Thread: Program to find the number of occurrences of words in the text file

Originally Posted by GregBrannon

Don't create a new Scanner object each time you need to get input. The object scan can be used both times.

I commented out the second scanner object "scan 1" now its not allowing me to enter an text its not listening here is the output

Enter the number of words to find its frequency:
3
Enter the file path to analyse the words:
Exception in thread "main" java.lang.NullPointerException
	at Counter.main(Counter.java:23)

Originally Posted by GregBrannon

I see that this line will most likely cause an error each time it is run:

if ( words[j + 1].equals( word ) )

You're not seeing that?

I think its not generating any error but still may be this could be the reason for not getting the output as i expect

Originally Posted by Norm

Can you post the new version of the code that shows the change you made when you commented out a statement? The change Greg was talking about meant that the code would use just one Scanner object. How does the new code do that?

import java.io.*;
import java.util.*;
 
class Counter {
 
    public static void main(String[] args) throws NullPointerException {
        System.out.println("Enter the number of words to find its frequency:");
        Scanner scan = new Scanner(System.in);
        int k = scan.nextInt();//Number of top words to be found is stored here
        //Scanner scan1 = new Scanner(System.in);
        System.out.println("Enter the file path to analyse the words:");
        String path = scan.nextLine();// Directory path will be stored here
        File folder = new File(path);
        File[] listOfFiles = folder.listFiles();//To get the list of file-names present in the path 
        BufferedReader br = null;
        String words[] = null;
        String line = "";
        String files;
        String word;
        int count = 0;
        // List<String> list = new ArrayList<String>();
        for (File f : listOfFiles) {
            if (f.isFile()) {
                files = f.getName();
                if (files.endsWith(".txt") || files.endsWith(".TXT")) {
                    try {
                        br = new BufferedReader(new FileReader(f));
                        try {
                            while ((line = br.readLine()) != null) {
                                line = line.toLowerCase(); // convert to lower case 
                                words = line.split("\\s+");
                                System.out.println("an ID "+ java.util.Arrays.toString(words));
                            }
                            java.util.Arrays.sort(words);
 
                            for (int i = 0; i < words.length; i++) {
                                word = words[i];
                                for (int j = 0; j < words.length; j++) {
                                    if (words[j + 1].equals(word)) {
                                        count++;
                                    }
 
                                }
                            }
                            System.out.println(count);
 
                        } catch (IOException e) {
                            System.out.println("I am sorry:" + e);
                        }
                    } catch (FileNotFoundException e) {
                        System.out.println("Here i have got" + e.getMessage());
                    }
 
                }
            }
        }
    }
}

Okay i have revoked the change in scanner object and i have added the println statement after splitting the contents by space (Line number:33)Norm

sorry i will follow this method of posting right from now thanks for being patient with my mistakes and here is the code which i am having now

import java.io.*;
import java.util.*;
 
class Counter {
 
    public static void main(String[] args) throws NullPointerException {
        System.out.println("Enter the number of words to find its frequency:");
        Scanner scan = new Scanner(System.in);
        int k = scan.nextInt();//Number of top words to be found is stored here
        Scanner scan1 = new Scanner(System.in);
        System.out.println("Enter the file path to analyse the words:");
        String path = scan1.nextLine();// Directory path will be stored here
        File folder = new File(path);
        File[] listOfFiles = folder.listFiles();//To get the list of file-names present in the path 
        BufferedReader br = null;
        String words[] = null;
        String line = "";
        String files;
        String word;
        int count = 0;
        // List<String> list = new ArrayList<String>();
        for (File f : listOfFiles) {
            if (f.isFile()) {
                files = f.getName();
                if (files.endsWith(".txt") || files.endsWith(".TXT")) {
                    try {
                        br = new BufferedReader(new FileReader(f));
                        try {
                            while ((line = br.readLine()) != null) {
                                line = line.toLowerCase(); // convert to lower case 
                                words = line.split("\\s+");
                                System.out.println("an ID "+ java.util.Arrays.toString(words));
                            }
                            java.util.Arrays.sort(words);
 
                            for (int i = 0; i < words.length; i++) {
                                word = words[i];
                                for (int j = 0; j < words.length; j++) {
                                    if (words[j + 1].equals(word)) {
                                        count++;
                                    }
 
                                }
                            }
                            System.out.println(count);
 
                        } catch (IOException e) {
                            System.out.println("I am sorry:" + e);
                        }
                    } catch (FileNotFoundException e) {
                        System.out.println("Here i have got" + e.getMessage());
                    }
 
                }
            }
        }
    }
}

here is the output of it :

Enter the number of words to find its frequency:
3
Enter the file path to analyse the words:
/home/dinesh/Desktop
an ID [hi]
an ID [hello]
an ID [hi]
an ID [hello]
an ID [hi]
an ID [norm]
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
	at Counter.main(Counter.java:40)

i have made the code to store the contents of line variable as an string array (words) by using space as delimiter and i am printing the contents of that words array,I have not made the code as to print the contents of the line variable here is that part of code Norm.

while ((line = br.readLine()) != null) {
                                line = line.toLowerCase(); // convert to lower case 
                                words = line.split("\\s+");
                                System.out.println("an ID "+ java.util.Arrays.toString(words));
                            }

After adding the printing statement to print the line variable contents my output is as follows:

Enter the number of words to find its frequency:
3
Enter the file path to analyse the words:
/home/dinesh/Desktop
hi
an ID [hi]
hello
an ID [hello]
hi
an ID [hi]
hello
an ID [hello]
hi
an ID [hi]
norm
an ID [norm]
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
	at Counter.main(Counter.java:41)

YES Norm you are right i haven't designed the steps anyway i am listing it below correct me if i go wrong here
1)When the user runs my application he must be prompted to enter the number say "n" which is used for pointing top "n" occurring words (i.e.,the words with higher frequency if n=5 then the code must display the 5 highly frequent words).

2)The code must prompt the user to enter the path from which the text files has to be read and find the frequency of words.

3)I am storing the contents of the file as an string array since we i can use Arrays.sort() method and sort its contents so it will be easy for me to find the frequency using indexof() method.<Note:This is not yet implemented>

4)Using looping to add each of the contents of the array and using equals() method to check whether the values are same and if so the count will be incremented.

5)Now after all the checking and counting i need to display the output as word:count in decreasing order of count value<Note:Not yet Implemented>

Norm,here i have modified the code now i have decided to read the file contents and add it to an List(ArrayList) by splitting the contents by space and i am using the inbuilt function of collection (sort) to sort the contents of the array list and i am trying to use the frequency function to count the frequency of words here is the updated version of code

import java.io.*;
import java.util.*;
 
class Counter {
 
    public static void main(String[] args) throws NullPointerException {
        System.out.println("Enter the number of words to find its frequency:");
        Scanner scan = new Scanner(System.in);
        int k = scan.nextInt();//Number of top words to be found is stored here
        Scanner scan1 = new Scanner(System.in);
        System.out.println("Enter the file path to analyse the words:");
        String path = scan1.nextLine();// Directory path will be stored here
        File folder = new File(path);
        File[] listOfFiles = folder.listFiles();//To get the list of file-names present in the path 
        BufferedReader br = null;
        String words[] = null;
        String line = "";
        String files;
        String word;
        int count = 0;
        List<String> list = new ArrayList<String>();
        for (File f : listOfFiles) {
            if (f.isFile()) {
                files = f.getName();
                if (files.endsWith(".txt") || files.endsWith(".TXT")) {
                    try {
                        br = new BufferedReader(new FileReader(f));
                        try {
                            while ((line = br.readLine()) != null) {
                                line = line.toLowerCase(); // convert to lower case.
                                words = line.split("\\s+");
                                list.addAll(java.util.Arrays.asList(words));
                                java.util.Collections.sort(list);
                                System.out.println(list);
                                /*for (String read : list) {
                                 System.out.println(Collections.frequency(list, read) + ":" + list);
                                 }*/
                            }
 
                            for (int i = 0; i < words.length; i++) {
                                word = words[i];
                                for (int j = 0; j < words.length; j++) {
                                    if (words[j + 1].equals(word)) {
                                        count++;
                                    }
 
                                }
                            }
                            System.out.println(count);
 
                        } catch (IOException e) {
                            System.out.println("I am sorry:" + e);
                        }
                    } catch (FileNotFoundException e) {
                        System.out.println("Here i have got" + e.getMessage());
                    }
 
                }
            }
        }
    }
}

But still the code is not working as i expected here is the latest output:

Enter the number of words to find its frequency:
3
Enter the file path to analyse the words:
/home/dinesh/Desktop
[hello, hello, hi, hi, hi, norm]
[, hello, hello, hi, hi, hi, norm]
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
	at Counter.main(Counter.java:44)

The statement "if (words[j + 1].equals(word))" is causing the error but if the new version is good i will remove this statements but please help me to fix the code.[COLOR="Silver"]

If the statement:

if (words[j + 1].equals(word))

is causing the error (as it was in your first post), then ensure that j + 1 does not exceed the array's number of elements - 1. How could you do that? By reducing what j can be by 1. Change the upper limit of j in the for loop statement.

Originally Posted by GregBrannon

If the statement:

if (words[j + 1].equals(word))

is causing the error (as it was in your first post), then ensure that j + 1 does not exceed the array's number of elements - 1. How could you do that? By reducing what j can be by 1. Change the upper limit of j in the for loop statement.

now i have replaced j+1 with j now no exception is thrown.
Kindly check my previous post i have modified the logic now using ArrayList and Collections.frequency function to count the frequency of the words but still its not working perfectly.

And what does "not working perfectly" mean? Show a sample run as you've done before and what is incorrect with it.

Can you see that the correct answer is there, but the output is "wrong"? And the output isn't really wrong, but it includes test code that you used to improve your program. I think you can remove the undesired output and then modify the actual output so that it produces what you need.

Originally Posted by Norm

It's not clear yet when the counting is supposed to be done:
Is the counting supposed to be done AFTER ALL the files have been read and all their contents stored in the arraylist?


Please post the steps in the design. What you have posted so far is too high level to use to write the code.

Here is the design
1)The application must prompt the user to enter the number "n" which is the top frequent words if n=2 the code must show the top two words with largest count as a result.

2)The user must be prompted to enter the file path from which the (.txt) files will be read.

3)The code must read all the .txt files one by one(if there exist many files) and addall the contents of those files into the ArrayList and they has to be splitted based on the space as delimiter.

4)Now the contents of the ArrayList has to be sorted using collection.sort() method.

5)Now the frequency of all the words in the ArrayList has to be computed using collection.frequency() method.

6)If n=2 then the two words with highest count will be printed in the console and the program needs to be terminated.

These are the steps i need to implement i am sorry i couldnt express the design i think

Originally Posted by Norm

Does that say that ALL the words from all the files are saved in the arraylist BEFORE the arraylist is sorted and searched?

Yes it must read ALL THE WORDS from ALL the files present in the path and add them all to the ArrayList BEFORE sorting.

Where is the collection class defined? There is a Collection class in java SE. The spelling is important: c is not the same as C.

I am sorry forgot to press "Shift" while typing "C" yes its built-in class "Collection"

What is the variable n? How is it given a value? Where is it stored?

I used the variable "k" instead of "n" in the code i have used Scanner object "scan" to get the input from the user and its value will be assigned to the variable "k".

Originally Posted by Norm

When describing the steps the program should take, use common words to describe a value, not a program variable name. What values does "n" hold? The frequency count of the current word? The number of files read?

The variable "n" has been replaced by "k".
k is used for storing an number which says the top occurring words i mean if hello:4,hi:3,norm:1 and the user enters 2 for "k" then hello:4,hi:3 is only displayed based on only the count of the words.

Since the arraylist contains all the words from all the files, what will the counting code do about repeated words?

What is the program's output now?

The code is to find the occurrences of the words in the ArrayList so we need to count the repeated words.