AlgoDaily - Find Duplicate Words

Good afternoon! Here's our prompt for today.

A very common interview challenge is to determine how often words appear in a given string or set of strings. Here's a version of this: return a list of duplicate words in a sentence.

For example, given 'The dog is the best', returns ["the"].

Likewise, given 'Happy thanksgiving, I am so full', you would return an empty array. This is because there are no duplicates in the string.

Constraints

The total length of the string <= 100000
The string will consist of ASCII characters (may be some or all)
Expected time complexity : O(n) where n are the number of words in the string
Expected space complexity : O(n)

Try to solve this here or in Interactive Mode.

How do I practice this challenge?

xxxxxxxxxx
 
var assert = require('assert');
​
function findDuplicates(str) {
  const dupes = [];
  // fill in
​
  return dupes;
}
​
try {
  assert.deepEqual(findDuplicates('The dog is the best'), ['the']);
​
  console.log(
    'PASSED: ' + "`findDuplicates('The dog is the best')` returns `['the']`"
  );
} catch (err) {
  console.log(err);
}
​
try {
  assert.deepEqual(findDuplicates('Happy thanksgiving, I am so full'), []);
​
  console.log(
    'PASSED: ' +
      "`findDuplicates('Happy thanksgiving, I am so full’)` returns `[]`"
  );
} catch (err) {
  console.log(err);
}

We'll now take you through what you need to know.

How do I use this guide?

The Foundation: Understanding the Problem

To find duplicate words in a sentence, we need a systematic approach for analyzing the occurrences of each word. Essentially, our task is to examine each word in the given string and keep a tab on how often it shows up.

Step 1: Tokenizing the Sentence into Words

What Does Tokenizing Mean?

Tokenization is the process of converting a sequence of text into individual "tokens" or units. In our context, this means splitting the given sentence into individual words.

How to Tokenize?

We can start by converting the entire string to lowercase to make our function case-insensitive. Then, we'll use the split function to break it into an array of words.

1let split_s = s.toLowerCase().split(' ');

xxxxxxxxxx
 
let s = "Original String";
let splitS = s.toLowerCase().split(" ");
let occurrences = {};
​
for (let word of splitS) {
  if (!occurrences.hasOwnProperty(word)) {
    occurrences[word] = 1;
  } else {
    occurrences[word]++;
  }
}
​
console.log(occurrences);

Step 2: Counting the Words Using a HashMap

Why a HashMap?

A HashMap provides an efficient way to store key-value pairs. In our case, the word will be the key, and its frequency will be the value. This will enable us to look up the frequency of a word in constant time.

How to Count?

We'll initialize an empty HashMap (or an object in JavaScript, or a dictionary in Python) to store the occurrences of each word. Let's call this HashMap occurrences.

Code Snippet for Counting

Here's how you can populate the occurrences HashMap:

1let occurrences = {};
2for(let word of split_s) {
3    if(word in occurrences) {
4        occurrences[word]++;
5    } else {
6        occurrences[word] = 1;
7    }
8}

Step 3: Identifying Duplicate Words

What are Duplicate Words?

Duplicate words are the keys in our HashMap with a value greater than 1, as these words have appeared more than once in the sentence.

How to Find Them?

We can loop through our occurrences HashMap to find these duplicates.

1let duplicates = [];
2for(let [word, count] of Object.entries(occurrences)) {
3    if(count > 1) {
4        duplicates.push(word);
5    }
6}

Step 4: Unveiling the Duplicates

Identifying Duplicates Made Easy

After painstakingly counting the frequency of each word, finding duplicates becomes a cakewalk. All we have to do is sift through our occurrences HashMap and pinpoint the words that have a frequency of 2 or more. These are our coveted duplicates.

Code Snippet for Identifying Duplicates

Here's how you can extract the duplicates from our occurrences HashMap:

1let duplicates = [];
2for (let [word, count] of Object.entries(occurrences)) {
3    if (count > 1) {
4        duplicates.push(word);
5    }
6}

Analyzing the Complexity of Our Final Solution

Time Complexity Revisited

Let n be the number of words in our input string s.

Populating the occurrences HashMap took O(n) time.
Traversing the occurrences HashMap to identify duplicates also takes O(n) time.

Summing these up, our algorithm works in linear O(n) time, which is remarkably efficient.

Space Complexity Revisited

We've used a HashMap (occurrences) to store the frequency of each word, and a list (duplicates) to store the duplicate words. In a worst-case scenario, each word in the sentence is unique, making our space complexity linear O(n).

xxxxxxxxxx
 
let s = "Original String Original String";
let split_s = s.toLowerCase().split(' ');
​
let occurrences = {};
​
for (let word of split_s) {
    if (!occurrences[word]) {
        occurrences[word] = 1;
    } else {
        occurrences[word] += 1;
    }
}
​
let dupes = [];
for (let k in occurrences) {
    if (occurrences[k] === 2) {
        dupes.push(k);
    }
}
​
console.log(dupes);

One Pager Cheat Sheet

This challenge requires returning a list of duplicate words in a sentence that consists of ASCII characters with a time complexity of O(n) and a space complexity of O(n).
We can use a hashmap (or JS object or Python dict) to store each word's frequency by looping through each one in the array and recording its occurrence.
We can easily find the duplicates by counting the occurrences of each word and looking for those with 2!

This is our final solution.

To visualize the solution and step through the below code, click Visualize the Solution on the right-side menu or the VISUALIZE button in Interactive Mode.

xxxxxxxxxx
}
 
var assert = require('assert');
​
function findDuplicates(str) {
  const dupes = [];
  const strLowerCase = str.toLowerCase();
  const strArr = strLowerCase.split(' ');
  const wordFreqCounter = {};
​
  strArr.forEach((word) => {
    if (!wordFreqCounter[word]) {
      wordFreqCounter[word] = 1;
    } else {
      wordFreqCounter[word] += 1;
    }
  });
​
  let allKeys = Object.keys(wordFreqCounter);
​
  allKeys.forEach((key) => {
    if (wordFreqCounter[key] > 1) {
      dupes.push(key);
    }
  });
​
  return dupes;
}
​
try {
  assert.deepEqual(findDuplicates('The dog is the best'), ['the']);
​
  console.log(
    'PASSED: ' + "`findDuplicates('The dog is the best')` returns `['the']`"

You're doing a wonderful job. Keep going!

If you had any problems with this tutorial, check out the main forum thread here.