Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 2 of 2

Thread: Program eating a lot of memory

  1. #1
    Junior Member
    Join Date
    Jan 2020
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Program eating a lot of memory

    Hello, i have a simple program that seems to take a lot of memory, about 700MB. As the program starts it keeps growing slowly until it reaches 700-1000MB and then it doesn't grow anymore. What my program does is open a webpage and analize its data, extract certain strings from its text and store it in a file. It stores each string in a separate line and the biggest of those files is about 4MB. I temporary store this strings in a set, manipulate it, compare it with others, read them from files and write them to files. The whole code is to big to share so i will just post some important things from it and hopefuly u will figure out where the problem is. Alternatively if there is a way for me to monitor which function or part of program is taking all this space, that would be ok too.

    This is how i read Strings from a file

    // fill positives	
    		Set<String> positives = new HashSet<String>();
     
    		try {
    			Scanner scanner = new Scanner(new File("positives.txt"));
    			while (scanner.hasNextLine()) {
     
    				String line = scanner.nextLine();														
    				positives.add(line); 
     
    			} 					
     
    			scanner.close();
    		} catch (FileNotFoundException e) {
    			e.printStackTrace();
    		}


    This is how i write Strings from a Set to file
    // write positives to file		
    		f1 = new FileWriter("positives.txt");
    		newLine = System.getProperty("line.separator");	
     
    		for (String s : positives) {
    			f1.write(s + newLine);
    		}
     
    		f1.close();


    This is how i remove files from set that are in the other set. Keep in mind that there is never more then 4MB of Strings in any of this set
    // remove from candidates all processed
     
    		candidates.removeAll(checkedTemp);

    This is how i get strings from a site.
    public static void readPage(String url, int pageNumber) {
     
     
     
    		String temp;
     
    		try {			
    			URL oracle;
    			oracle = new URL(url + String.valueOf(pageNumber));			
    			BufferedReader in = new BufferedReader(
    			new InputStreamReader(oracle.openStream()));
     
    			String inputLine;
    			int line= 0;
     
    			while ((inputLine = in.readLine()) != null) {
     
     
    					temp = extractStrings(inputLine);	
     
    					if(temp != null) {		
    						candidates.add(temp);											
    						candidatesCounter++;			
    					} 
    					line++;
     
     
    				}		
    					in.close();				
    			} catch(Exception e) {
    				System.out.print("error: ");
    				System.out.println(e);		
    			}	
     
     
     
     
     
     
    	}

    This is how i process the webpage. It ready about 40 pages full of text and uses StringUtils library to find strings that are in a certain place (and example would be, find all strings that start with /ri and end with .jpg)
    public static String extractStrings(String line) {				
     
    		String result = StringUtils.substringBetween(line, "ri/", ".jpg");
    		if(result != null) return result;
     
    		return null;     
     
    	}

  2. #2
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,140
    Thanks
    65
    Thanked 2,720 Times in 2,670 Posts

    Default Re: Program eating a lot of memory

    There are some monitoring programs that will help you find where the code is executing and perhaps where memory is being used. I don't know the names of any of them.
    If you don't understand my answer, don't ignore it, ask a question.

Similar Threads

  1. How to detect current memory free and used in java program ?
    By tanc08 in forum Java Theory & Questions
    Replies: 2
    Last Post: August 29th, 2014, 08:48 AM
  2. Memory test game program !
    By sharpedge in forum What's Wrong With My Code?
    Replies: 5
    Last Post: February 10th, 2013, 11:41 AM
  3. Properly releasing memory to avoid memory pileup/crash
    By fickletrick in forum What's Wrong With My Code?
    Replies: 6
    Last Post: July 22nd, 2012, 10:09 AM
  4. Java program runs out of memory (-xms1024 -xmx1024)
    By Jack Hammered in forum What's Wrong With My Code?
    Replies: 4
    Last Post: January 23rd, 2012, 10:29 AM
  5. HELP ME PLEASE MEMORY MANAGEMENT PROGRAM USING JAVA
    By lockwater in forum Java Theory & Questions
    Replies: 1
    Last Post: December 2nd, 2010, 08:30 AM