Coding N Concepts
How to Remove Duplicates from Array in JavaScript
Set(), Array.filter(), Array.forEach() and Array.reduce()
In this tutorial, we’ll learn how to remove duplicates from an Array in JavaScript and return only unique values.
Here is a quick view of different ways to remove duplicates from an Array:
// Array:
const fruits = ['apple', 'orange', 'orange', 'apple', 'banana', 'apple'];
// 1. ES6 Set
[...new Set(fruits)];
// 2. filter
fruits.filter((item, index) => fruits.indexOf(item) == index);
// 3. forEach
fruits.forEach(item => !uniqueFruits.includes(item) && uniqueFruits.push(item));
// 4. reduce
fruits.reduce((newarray, item) => newarray.includes(item) ? newarray : [...newarray, item], []);
// Result:
// ▶ Set(3) {"apple", "orange", "banana"}
Use ES6 Set()
If you have started using ES6 then it is recommended to use Set object.
Setobject lets you store unique values of any type, whether primitive values or object reference.
When you pass an array to new Set() object, it removes any duplicate values. Let’s look at the code where two things are going on:
- First, we create a new
Setobject by passing an array which removes the duplicates. - Second, we convert this object back to an array by using spread operator
...
const fruits = ['apple', 'orange', 'orange', 'apple', 'banana', 'apple'];
const uniqueSet = new Set(fruits);
// ▶ Set(3) {"apple", "orange", "banana"}
const backToArray = [...uniqueSet];
// ▶ (3) ["apple", "orange", "banana"]
Alternatively, you can also use Array.from to convert a Set into an array:
const fruits = ['apple', 'orange', 'orange', 'apple', 'banana', 'apple'];
const uniqueFruits = Array.from(new Set(fruits));
// ▶ Set(3) {"apple", "orange", "banana"}
Use Array.filter()
The filter() method filter out the elements from an array based on the condition we provide. In other words,
- if the condition returns true, it will be included in filtered array
- if the condition returns false, it will NOT be included in filtered array
const fruits = ['apple', 'orange', 'orange', 'apple', 'banana', 'apple'];
const uniqueFruits = fruits.filter((item, index) => fruits.indexOf(item) == index);
// ▶ Set(3) {"apple", "orange", "banana"}
Here use of Array.indexOf() method is very important which gives you the first occurrence of element in case there are duplicates. Let’s see the code:
const fruits = ['apple', 'orange', 'orange', 'apple', 'banana', 'apple'];
fruits.indexOf('apple'); //0
fruits.indexOf('orange'); //1
fruits.indexOf('banana'); //4
In the above case, our filter() method filters out the first occurrence of each unique element.
Use Array.forEach()
The forEach() method can also be used to loop through elements of an array and push() elements to the new array which is not already there.
const fruits = ['apple', 'orange', 'orange', 'apple', 'banana', 'apple'];
const uniqueFruits = [];
fruits.forEach(item => !uniqueFruits.includes(item) && uniqueFruits.push(item));
console.log(uniqueFruits);
// ▶ Set(3) {"apple", "orange", "banana"}
Use Array.reduce()
The reduce() method is used to reduce the elements of the array and combine them into a new array based on some reducer function that you pass.
In this case, our reducer function is checking if our new array contains the item. If it doesn’t, push that item into our new array. Otherwise, skip that element and return just our new array as is.
Reduce is always bit difficult to understand. Let’s look at the code:
const fruits = ['apple', 'orange', 'orange', 'apple', 'banana', 'apple'];
const uniqueFruits =
fruits.reduce((newarray, item) => newarray.includes(item) ? newarray : [...newarray, item], []);
console.log(uniqueFruits);
// ▶ Set(3) {"apple", "orange", "banana"}
Performance
We have created an array with 1 million random numbers (ranging from 1 to 1000). Let’s check the performance of each method.
// Array:
var numbers = [];
for ( var i = 0; i < 1000000; i++ ) {
numbers.push(Math.floor((Math.random() * 1000) + 1));
}
// 1. ES6 Set
console.time("set");
[...new Set(numbers)];
console.timeEnd("set");
// 2. filter
console.time("filter");
numbers.filter((item, index) => numbers.indexOf(item) == index);
console.timeEnd("filter");
// 3. forEach
console.time("forEach");
var uniqueNumbers = [];
numbers.forEach(item => !uniqueNumbers.includes(item) && uniqueNumbers.push(item));
console.timeEnd("forEach");
// 4. reduce
console.time("reduce");
numbers.reduce((newarray, item) => newarray.includes(item) ? newarray : [...newarray, item], []);
console.timeEnd("reduce");
Here is the result when i ran the above code on MacBook Pro’s Google Chrome v83.0.4103.106 (64-bit):
set: 23.051025390625ms
filter: 1004.9609375ms
forEach: 471.6630859375ms
reduce: 472.902099609375ms
We see that ES6 Set object is a clear winner followed by forEach loop. It is highly recommended to use ES6 Set to remove duplicates from an array because of its shorthand syntax and performance.
