Use the Remove Duplicates node to identify and delete items that are identical across all fields or a subset of fields. This is helpful in situations where you can end up with duplicate data, such as a user creating multiple accounts, or a customer submitting the same order multiple times. When working with large datasets it becomes more difficult to spot and remove these items.
- Compare: specify which fields of the input data Mosaic Workflows should compare to check if they're the same. The following options are available:
- All Fields: compares all fields of the input data.
- All Fields Except: enter which input data fields Mosaic Workflows should exclude from the comparison. You can provide multiple values separated by commas.
- Selected Fields: enter which input data fields Mosaic Workflows should include in the comparison. You can provide multiple values separated by commas.
- If you choose All Fields Except or Selected Fields, Mosaic Workflows displays Options > Add Field. Use this to add more optional settings, including:
- Disable Dot Notation: when disabled, you can't reference child fields (in the format
parent.child). - Remove Other Fields: keep the fields that you're comparing and remove the others.
- Disable Dot Notation: when disabled, you can't reference child fields (in the format
Learn more about data structure and data flow in Mosaic Workflows workflows.