Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 5 of 5

Thread: Can't read really big files

  1. #1
    Junior Member
    Join Date
    Sep 2012
    Posts
    3
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Can't read really big files

    Hey,

    So basically I build a code to check lines on a .txt file and those likes compare a substring and change it to another saved string (always in the same position). Everything worked like a charm, no problem whatsoever, the messed up situation pops when I try to run the code on txt files with more than 2 million lines. This is definitely a problem because I need to run it on files with more than 10 million lines. What I tried to do was to save it on an ArrayList but it does not work, so Im lost here, been with this issue for more than 3 days, now Im getting to the point of dying. Here's the method call for the proccess:

          public void checaFechaSegunLugarUnico(String path, String archivo, int inicio, int fin) throws FileNotFoundException{
                  jTextArea2.setText("");    
                  try{
                  FileReader fstream = new FileReader(path);
                  //DataInputStream in = new DataInputStream(fstream);
                  BufferedReader br = new BufferedReader(fstream);
                  String strLine,primero, segundo, pivote,aux;
                  int i = 0;
                  ArrayList  almacem;
                  almacem = new ArrayList();
                  primero= "";
                  jTextArea2.append("Hola\n");
                  boolean bandera=true;
                  while ((strLine = br.readLine()) != null){
                      almacem.add(strLine);
                      /*if(strLine.length() < fin){
                      aux= strLine;
                      jTextArea2.append(aux+"\n");
                      }
                      else{
                      pivote = strLine.substring(inicio,fin);
                      if(pivote.equals(primero)) {
                              aux= strLine;
                          }
                      else{
                      if (bandera == true){
                          primero = pivote;
                          bandera = false;
                          aux=strLine;
                      }  else{
     
                          aux = strLine.substring(0, inicio)+primero+strLine.substring(fin);
                          //aux = strLine.replaceAll(pivote, primero);
                      }
                      }
     
                      jTextArea2.append(aux+"\n");
     
                  */}
                      jTextArea2.append("Salio\n");  
                      jTextArea2.append("Total leído: "+ almacem.size());
                  //}
                  /*SimpleDateFormat format = new SimpleDateFormat("yy-M-d");
                  jTextArea1.append(primero+"\n");
                  Date check = format.parse(primero);
                  Calendar calendar = Calendar.getInstance();
                  calendar.setTime(check);
                  int dia = Integer.parseInt(primero.substring(8));
                  int mes = calendar.get(Calendar.MONTH);
                  int año = calendar.get(Calendar.YEAR);
                  int ultDiaHab= obtenerUltimoDia(mes,año);
                  if(dia == ultDiaHab){
                      jTextArea1.append("La fecha capturada en el archivo "+archivo+" si concuerda con el último dia habil del mes\n");
                      crearArchivo(path,archivo);
                      jTextArea2.setText("");
                  }
                  else {
                      jTextArea1.append("La fecha capturada en el archivo "+archivo+" no concuerda con el último dia habil del mes\n");
                      jTextArea2.setText("");
                  }*/
     
                  }catch (Exception e){
     
                  }
     
       }

    Any help will be greatly appreciated.


  2. #2
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,165
    Thanks
    65
    Thanked 2,725 Times in 2,675 Posts

    Default Re: Can't read really big files

    it does not work,
    Please explain what "does not work" means. If you are getting error messages, copy and paste here the full text of the error message.
    If you don't understand my answer, don't ignore it, ask a question.

  3. The Following 2 Users Say Thank You to Norm For This Useful Post:

    curmudgeon (September 3rd, 2012), Yoso14 (September 3rd, 2012)

  4. #3
    Super Moderator curmudgeon's Avatar
    Join Date
    Aug 2012
    Posts
    1,130
    My Mood
    Cynical
    Thanks
    64
    Thanked 140 Times in 135 Posts

    Default Re: Can't read really big files

    I second what Norm says. Also this type of code makes me nervous:

                  }catch (Exception e){
     
                  }

    Please don't ignore exceptions. At least print a stacktrace:


                  }catch (Exception e){
                      e.printStackTrace();  // *** added *** 
                  }

  5. The Following 2 Users Say Thank You to curmudgeon For This Useful Post:

    Norm (September 3rd, 2012), Yoso14 (September 3rd, 2012)

  6. #4
    Junior Member
    Join Date
    Sep 2012
    Posts
    3
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default Re: Can't read really big files

    First of all, thanks for your response. Now, Im supposed to get the size of the ArrayList almacem in jTextArea2 because I append the String there with jTextArea2.append("The size is " + almacem.size()) or at least I should get the first String that says jTextArea2.append("Salio\n"), but I aint getting any of those two in the Text Area, so my guessing is that the arrayList is not getting filled with the strLine values.

  7. #5
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    25,165
    Thanks
    65
    Thanked 2,725 Times in 2,675 Posts

    Default Re: Can't read really big files

    Add some println statements to print out the status of the execution as the lines are read.
    Print a message every nn (try different values for nn: 100 or 1000 etc) lines by define an int variable: cntr that is incremented by 1 for each line read and use the % to control when it prints:
    if(cntr++ % nn == 0) System.println("cntr=" + cntr);

    Was there an error message printed after you added the call to printStackTrace() to the catch block?
    If you don't understand my answer, don't ignore it, ask a question.

Similar Threads

  1. Replies: 3
    Last Post: August 3rd, 2012, 10:40 AM
  2. How can I read txt files in java program??
    By sakura_smile in forum What's Wrong With My Code?
    Replies: 5
    Last Post: June 9th, 2012, 06:18 PM
  3. [SOLVED] Using Scanner to read in text files
    By PeskyToaster in forum File I/O & Other I/O Streams
    Replies: 0
    Last Post: April 16th, 2012, 04:57 PM
  4. Read input, read file, find match, and output... URGENT HELP!
    By MooseHead in forum What's Wrong With My Code?
    Replies: 3
    Last Post: April 3rd, 2012, 11:01 AM
  5. How to read from all files in a directory & plot the contents?
    By 123 in forum File I/O & Other I/O Streams
    Replies: 1
    Last Post: February 11th, 2010, 12:43 PM