Removing Duplicates in Arrays in JavaScript: Various Methods

In software development, it’s often necessary to remove duplicate elements from arrays. JavaScript provides several methods to achieve this, depending on whether you want to remove duplicates based on one or multiple properties or retain the last duplicate element. This various ways to remove duplicates from arrays using the filter() method and other approaches.

1. Removing Duplicates Based on Unique ID

If you have an array of objects with a unique ID property, you can remove duplicates based on that ID. This method ensures that only one object with the same ID remains in the array.

Example Code:

const arr = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 1, name: 'Alice' },
  { id: 3, name: 'Charlie' }
];

const uniqueById = arr.filter((v, i, a) => a.findIndex(t => t.id === v.id) === i);

console.log(uniqueById);
// Output: [ { id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' } ]

Explanation:

• findIndex(t => t.id === v.id): Finds the first index of an element with the same ID.
• a.findIndex(t => t.id === v.id) === i: Ensures that only the first occurrence of the element is retained.

2. Removing Duplicates Based on Multiple Properties

If you need to remove duplicates based on multiple properties, such as place and name, you can use a similar approach by checking the combination of these properties.

Example Code:

const arr = [
  { id: 1, place: 'Park', name: 'Alice' },
  { id: 2, place: 'Library', name: 'Bob' },
  { id: 1, place: 'Park', name: 'Alice' },
  { id: 3, place: 'Café', name: 'Charlie' }
];

const uniqueByPlaceAndName = arr.filter((v, i, a) => a.findIndex(t => t.place === v.place && t.name === v.name) === i);

console.log(uniqueByPlaceAndName);
// Output: [ { id: 1, place: 'Park', name: 'Alice' }, { id: 2, place: 'Library', name: 'Bob' }, { id: 3, place: 'Café', name: 'Charlie' } ]

Explanation:

• findIndex(t => t.place === v.place && t.name === v.name): Finds the first index of an element with the same combination of place and name.

3. Removing Duplicates Based on All Properties

If you want to remove duplicates based on all properties in an object, you can convert the objects to JSON strings for comparison. However, this method can be slow for large arrays.

Example Code:

const arr = [
  { id: 1, name: 'Alice', age: 25 },
  { id: 2, name: 'Bob', age: 30 },
  { id: 1, name: 'Alice', age: 25 },
  { id: 3, name: 'Charlie', age: 35 }
];

const uniqueByAllProperties = arr.filter((v, i, a) => a.findIndex(t => JSON.stringify(t) === JSON.stringify(v)) === i);

console.log(uniqueByAllProperties);
// Output: [ { id: 1, name: 'Alice', age: 25 }, { id: 2, name: 'Bob', age: 30 }, { id: 3, name: 'Charlie', age: 35 } ]

Explanation:

• JSON.stringify(t) === JSON.stringify(v): Converts objects to JSON strings and compares these strings.

4. Retaining the Last Occurrence

If you want to keep the last occurrence of an element with duplicates, you can reverse the array, remove duplicates, and then reverse it back.

Example Code:

const arr = [
  { id: 1, name: 'Alice' },
  { id: 2, name: 'Bob' },
  { id: 1, name: 'Alice' },
  { id: 3, name: 'Charlie' }
];

const keepLastOccurrence = arr.slice().reverse().filter((v, i, a) => a.findIndex(t => t.id === v.id) === i).reverse();

console.log(keepLastOccurrence);
// Output: [ { id: 2, name: 'Bob' }, { id: 3, name: 'Charlie' }, { id: 1, name: 'Alice' } ]

Explanation:

• slice().reverse(): Reverses the array.
• filter(): Removes duplicates based on ID by checking the last occurrence.
• reverse(): Reverses the array back to its original order.

5. Conclusion

Managing duplicates in JavaScript arrays can be done using various methods depending on the criteria for removing duplicates. Using the filter() method, you can easily remove duplicates based on one or multiple properties or retain the last occurrence. Choose the method that best suits your application’s needs to ensure your data remains clean and consistent.