Page "Extract, transform, load" Paragraph 54

Whether to do certain operations in the database or outside may involve a trade-off.

For example, removing duplicates using < tt > distinct </ tt > may be slow in the database ; thus, it makes sense to do it outside.

On the other side, if using < tt > distinct </ tt > will significantly ( x100 ) decrease the number of rows to be extracted, then it makes sense to remove duplications as early as possible in the database before unloading data.