[issue40347] Programming FAQ about "How do you remove duplicates from a list?" -- Improve the examples + Mention possible caveats

[Dominik V.]

New submission from Dominik V. <dominik.vilsmeier1123 at gmail.com>:

https://docs.python.org/3/faq/programming.html#how-do-you-remove-duplicates-from-a-list

In the beginning it points to the recipes at https://code.activestate.com/recipes/52560/ which does mention various caveats such as

> [...] whether [elements are] hashable, and whether they support full comparisons.

It then shows a concrete example implementation which however does require that the elements define a total ordering. The code for the example is pretty long so it might discourage new programmers before they even discover the most likely best solution which comes at the end of the section:

    list(set(mylist))

This seems by far the most useful solution with evidence from this StackOverflow question: https://stackoverflow.com/questions/7961363/removing-duplicates-in-lists

Hence I propose two changes:

1. Include the first sentence of the abstract from the recipes at https://code.activestate.com/recipes/52560/ in the FAQs: "The fastest way to remove duplicates from a sequence depends on some pretty subtle properties of the sequence elements, such as whether they're hashable, and whether they support full comparisons." at the beginning in order to mention possible caveats.
2. Either remove or move the code example relying on `sort` in order to give more visibility to the most likely more relevant solution using `set`. In any case it has the disclaimer about hashability and hence won't trick people into believing it works for all cases.

If the `sort` example is not removed, at least it's description should mention that elements must define a total ordering (e.g. if the elements are sets it won't generally work).

----------
assignee: docs at python
components: Documentation
messages: 366893
nosy: Dominik V., docs at python
priority: normal
severity: normal
status: open
title: Programming FAQ about "How do you remove duplicates from a list?" -- Improve the examples + Mention possible caveats
type: enhancement
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40347>
_______________________________________