Remove Duplicate Strings from a List in Python

Dealing with duplicate strings in a list is a common task in Python, often come across this when working with string/text data. In this article, we’ll explore multiple approaches to remove duplicate strings from a list in Python.

Below are some of the ways by which we can remove duplicate strings from a list in Python:

Using a set() Function
Using List Comprehension
Using OrderedDict.fromkeys() Method
Using Counter() from collections module

Remove Duplicate Strings Using a set function()

This approach utilizes the unique property of sets to automatically remove duplicate elements, resulting in a list with only unique strings.

Python

# Sample list with duplicate strings
original_list = ['apple', 'orange', 'banana', 'apple', 'grape', 'banana']
 
# Remove duplicates using a set
unique_list = list(set(original_list))
 
print(unique_list)

Output

['orange', 'grape', 'apple', 'banana']

Remove Duplicate Strings Using List Comprehension

In this approach, list comprehension is used to iterate through the original list and append only non-duplicate items to the new list.

Python

# Sample list with duplicate strings
original_list = ['apple', 'orange', 'banana', 'apple', 'grape', 'banana']
 
# Remove duplicates using list comprehension
unique_list = []
[unique_list.append(item) for item in original_list if item not in unique_list]
 
print(unique_list)

Output

['apple', 'orange', 'banana', 'grape']

Remove Duplicate Strings Using OrderedDict.fromkeys() Function

The OrderedDict.fromkeys() method is utilized to create a dictionary without duplicates, and the result is converted back to a list.

Python

# Sample list with duplicate strings
original_list = ['apple', 'orange', 'banana', 'apple', 'grape', 'banana']
 
# Remove duplicates using OrderedDict.fromkeys()
unique_list = list(dict.fromkeys(original_list))
 
print(unique_list)

Output

['orange', 'grape', 'apple', 'banana']

Remove Duplicate Strings Using Counter() Method

The Counter class from the collections module is employed to count the occurrences of each element, and the keys() method provides the unique elements in the original order.

Python

from collections import Counter
 
# Sample list with duplicate strings
original_list = ['apple', 'orange', 'banana', 'apple', 'grape', 'banana']
 
# Remove duplicates using Counter
unique_list = list(Counter(original_list).keys())
 
print(unique_list)

Output

['orange', 'grape', 'apple', 'banana']

Reffered: https://www.geeksforgeeks.org

Python

Related
R vs Python: Which is Easier to Learn
Append a Key to a Python Dictionary
Pass Arguments to the Metaclass from the Class in Python
Multiplication Table Using While Loop in Python
Add Values into an Empty List from Python For Loop

Type:	Geek
Category:	Coding
Sub Category:	Tutorial
Uploaded by:	Admin
Views:	13