Thread: How do I write output from one program to several files?

Originally Posted by Norm

Please copy the full text of the error message and paste it here.
Look at the line where the NPE happens, find the variable with the null value and backtrack in the code to see why that variable does not have a valid value.


The code is poorly formatted and that makes it hard to read and understand.
The {s and }s are not correctly positioned so the nesting of logic can be seen.

Here is the full text of the error message:
Moving through jobs one by one with a loop:
Job[id: 0, Size: 301, Platform: Windows, Priority: High]
Job[id: 1, Size: 115, Platform: Windows, Priority: Medium]
Job[id: 2, Size: 229, Platform: Windows, Priority: Low]
Job[id: 3, Size: 915, Platform: Mac, Priority: Medium]
Job[id: 4, Size: 811, Platform: Mac, Priority: High]
Job[id: 5, Size: 57, Platform: Mac, Priority: High]
Job[id: 6, Size: 965, Platform: Windows, Priority: Low]
Job[id: 7, Size: 767, Platform: Mac, Priority: Medium]
Job[id: 8, Size: 258, Platform: Solaris, Priority: High]
Job[id: 9, Size: 598, Platform: Mac, Priority: Medium]
Job[id: 10, Size: 771, Platform: Solaris, Priority: VeryHigh]
Job[id: 11, Size: 109, Platform: Mac, Priority: High]
Job[id: 12, Size: 547, Platform: Solaris, Priority: VeryHigh]
Job[id: 13, Size: 548, Platform: Windows, Priority: High]
Job[id: 14, Size: 123, Platform: Windows, Priority: High]
Job[id: 15, Size: 203, Platform: Mac, Priority: Medium]
Job[id: 16, Size: 226, Platform: Mac, Priority: Medium]
Job[id: 17, Size: 687, Platform: Windows, Priority: High]
Job[id: 18, Size: 985, Platform: Mac, Priority: VeryHigh]
Job[id: 19, Size: 113, Platform: Mac, Priority: VeryHigh]
Job[id: 20, Size: 821, Platform: Windows, Priority: VeryHigh]
Job[id: 21, Size: 389, Platform: Mac, Priority: VeryHigh]
Job[id: 22, Size: 395, Platform: Solaris, Priority: Low]
Job[id: 23, Size: 80, Platform: Solaris, Priority: Low]
Job[id: 24, Size: 997, Platform: Windows, Priority: High]
Job[id: 25, Size: 171, Platform: Windows, Priority: High]
java.lang.NullPointerException
Job[id: 26, Size: 56, Platform: Solaris, Priority: High]
Job[id: 27, Size: 830, Platform: Windows, Priority: VeryHigh]
Job[id: 28, Size: 917, Platform: Mac, Priority: High]
Job[id: 29, Size: 106, Platform: Mac, Priority: VeryHigh]
Creating a dispatcher
Moving through jobs one by one with a loop:
Getting all jobs with priority VeryHigh
Job[id: 10, Size: 771, Platform: Solaris, Priority: VeryHigh]
at utilities.FileOutput.print(FileOutput.java:53)
Job[id: 12, Size: 547, Platform: Solaris, Priority: VeryHigh]
at utilities.FileOutput.println(FileOutput.java:63)
Job[id: 18, Size: 985, Platform: Mac, Priority: VeryHigh]
at SendToOutputFile.main(SendToOutputFile.java:113)
Job[id: 19, Size: 113, Platform: Mac, Priority: VeryHigh]
java.lang.NullPointerException
at utilities.FileOutput.print(FileOutput.java:53)
at utilities.FileOutput.println(FileOutput.java:63)
at SendToOutputFile.main(SendToOutputFile.java:113)
java.lang.NullPointerException
Job[id: 20, Size: 821, Platform: Windows, Priority: VeryHigh]
Job[id: 21, Size: 389, Platform: Mac, Priority: VeryHigh]
Job[id: 27, Size: 830, Platform: Windows, Priority: VeryHigh]
at utilities.FileOutput.print(FileOutput.java:53)
Job[id: 29, Size: 106, Platform: Mac, Priority: VeryHigh]
Getting all jobs with priority High
Job[id: 0, Size: 301, Platform: Windows, Priority: High]
Job[id: 4, Size: 811, Platform: Mac, Priority: High]
at utilities.FileOutput.println(FileOutput.java:63Job[id: 5, Size: 57, Platform: Mac, Priority: High]
)
Job[id: 8, Size: 258, Platform: Solaris, Priority: High]
at SendToOutputFile.main(SendToOutputFile.java:113)
Job[id: 11, Size: 109, Platform: Mac, Priority: High]
Job[id: 13, Size: 548, Platform: Windows, Priority: High]
Job[id: 14, Size: 123, Platform: Windows, Priority: High]
Job[id: 17, Size: 687, Platform: Windows, Priority: High]
Job[id: 24, Size: 997, Platform: Windows, Priority: High]
Job[id: 25, Size: 171, Platform: Windows, Priority: High]
Job[id: 26, Size: 56, Platform: Solaris, Priority: High]
Job[id: 28, Size: 917, Platform: Mac, Priority: High]
Getting all jobs with priority Medium
Job[id: 1, Size: 115, Platform: Windows, Priority: Medium]
Job[id: 3, Size: 915, Platform: Mac, Priority: Medium]
Job[id: 7, Size: 767, Platform: Mac, Priority: Medium]
Job[id: 9, Size: 598, Platform: Mac, Priority: Medium]
Job[id: 15, Size: 203, Platform: Mac, Priority: Medium]
Job[id: 16, Size: 226, Platform: Mac, Priority: Medium]
Getting all jobs with priority Low
Job[id: 2, Size: 229, Platform: Windows, Priority: Low]
BUILD SUCCESSFUL (total time: 0 seconds)
Note: No output is going to the files, all the output here are on the screen.
Backtracking:
(1) @ at utilities.FileOutput.print(FileOutput.java:53) the code is : bw.write(str);
(2) @ at utilities.FileOutput.println(FileOutput.java:63) the code is: print(str);
(3) @ at SendToOutputFile.main(SendToOutputFile.java:113), the code is: fileD.println(jobAsString, true);
Hope this gives you enough insight to the problem!

--- Update ---

In addition to the above, the number one error @ at utilities.FileOutput.print (FileOutput.java:53) is a method with the follwing code:

 <public void print(String str) throws IOException 
 
    {
        bw.write(str);
 
    }
>

Happy to inform that I ve got everything running the way I want now. But I need to post a new thread I have two more questions and I shall be done:
(1) Is how to read from a file
(2) How to share jobs between threads

--- Update ---

Originally Posted by Norm

There were several error messages mixed in with the program's output.
In this one there was a varible with a null value at line 113. Look at line 113, find the variable that had a null value and backtrack in the code to see why that variable did not have a valid value.


Does the code follow these suggestions?
Did you see the steps I suggested for the program in post#15?
Here they are again:
Open some output files to write items to<<<<The files are all opened BEFORE the loop
loop through a list of items
write item to one of the files based on its content
end loop
close files <<<<<<<<<< The close is after the loop


It looks like the opens and closes are inside the loop???

Yes, I followed your suggetsion, thanks, it helped greatly.

Originally Posted by Norm

The Scanner class has methods for reading from text files.


Can you explain why you want to use threads.
What do you mean by "job"
what is to be shared?

I have a thread class that uses threads to generate these jobs, in that class, a range of jobs from 1,000 jobs upto 1,000000 in steps of 2,000 is defined to be generated and sorted. Each set of jobs will be executed by 1 thread, 2 threads, 3 threads and 4 threads. The problem am encountering is that the jobs are not being shared by the threads as each thread in each group is doing the same job and they are even not synchronized. What this mean is that instead of say 2 threads to share 1000 jobs, each of the two thread is executing same thing individually. What that mean is that 2000 jobs are executed by 2 threads instead of 1000 jobs shared between them

Originally Posted by Norm

Is this "job" some data being written to a file
or is it the execution of some code?
The use of the word jobs is ambiguous for me.

How does the code control which thread gets which "job"? Is there a single source for a "job" that each thread can call to get the next "job"?

This set of output am currently writing to output files are exactly what I need to do with the threads. I mean the last problem that just got solved

Originally Posted by Norm

Post the code that you are having problems using threads in and explain what the threads are supposed to do.
The last problem had nothing to do with trying to do parallel processing using threads.

Here is my thread code, I intend to split the task of generating the jobs in my previous question among different number of threads ranging from 1 to 4.
The number of the jobs to be generated and writing to those files are specified in the former file called SeveralJobs (which we just got done earlier). What I need do is to share the jobs among the threads equally: Am also to record the time used for doing each set of the execution by each set of thread

 
import java.util.LinkedList;
import java.util.Random;
 
/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
 
/**
 *
 * @author PC
 */
public class ThreadingExample extends TestRuns
{
    static final int NUMBER_OF_THREADS = 4;
 
 
 
   // public static void main(String[] args)
    {
        // create a list of things to do
        ThreadingExample.Work workToDo = new ThreadingExample.Work();
        workToDo.put(new ThreadingExample.PieceOfWork("piece 1"));
        workToDo.put(new ThreadingExample.PieceOfWork("piece 2"));
        workToDo.put(new ThreadingExample.PieceOfWork("piece 3"));
        workToDo.put(new ThreadingExample.PieceOfWork("piece 4"));
 
        // create threads
        ThreadingExample.WorkingThread[] threads = new ThreadingExample.WorkingThread[NUMBER_OF_THREADS];
        for(int i=0; i<threads.length; i++)
        {
            threads[i] = new ThreadingExample.WorkingThread(workToDo, "Thread " + i);
        }
        long startTime = System.currentTimeMillis();
 
        // start threads
        for(int i=0; i<threads.length; i++)
        {
            threads[i].start();
        }
 
        // wait for the list of things to do to be empty
        while(!workToDo.isDone())
        {
            try
            {
                Thread.sleep(0, 1);
            } catch(Exception e)
            {
 
            }
        }
 
        // wait for the last thread to finish
        while(true)
        {
            boolean finished = true;
 
            for(ThreadingExample.WorkingThread thread : threads)
            {
                if(thread.isWorking())
                    finished = false;
            }
 
            if(finished)
                break;
 
        }
 
        long endTime = System.currentTimeMillis();
 
        log("done in "+ (endTime - startTime) + "ms");
 
        for(ThreadingExample.WorkingThread thread : threads) {
            thread.cancel();
        }
    }
 
 
    static class WorkingThread extends Thread
    {
        private final ThreadingExample.Work pWork;
        private final String pThreadName;
 
        private boolean pRequestedStop = false;
        private boolean pIsWorking = true;
 
        public boolean isWorking()
        {
            return pIsWorking;
        }
 
        public WorkingThread(ThreadingExample.Work workspace, String name)
        {
            pWork = workspace;
            pThreadName = name;
        }
 
        /**
         * Request to stop this thread after completing current piece of work.
         */
        public void cancel()
        {
            pRequestedStop = true;
        }
 
        @Override
        public void run()
        {
            System.out.println("Starting thread " + pThreadName);
 
            // run indefinetely
            while(!pRequestedStop)
            {
                ThreadingExample.PieceOfWork piece = pWork.pop(1);
 
                if(piece != null)
                {
                    piece.doWork(pThreadName);
                }
 
                pIsWorking = !pWork.isDone();
            }
        }
    }
 
    static class Work 
    { 
        private final LinkedList<ThreadingExample.PieceOfWork> pList = new LinkedList<>();;
 
        public synchronized void put(ThreadingExample.PieceOfWork piece)
        {
            synchronized(pList)
            {
                pList.add(piece);
                pList.notify();
            }
        }
 
        public synchronized ThreadingExample.PieceOfWork pop(long timeout)
        {
            synchronized(pList)
            {
                try
                {
                    pList.wait(timeout);
                } 
                catch(InterruptedException e){}
 
                if(pList.size() == 0)
                    return null;
                else
                    return pList.removeFirst();
            }
        }
 
        public boolean isDone()
        {
            synchronized(pList)
            {
                return pList.isEmpty();
            }
        }
    }
 
    static class PieceOfWork
 
    {
 
        private String pDescripion;
 
 
        public PieceOfWork(String description)
        {
            pDescripion = description;
        }
 
         private void work()
        {
 
           long StartTime = System.currentTimeMillis();
 
           SeveralJobs severalJobs = new SeveralJobs(); 
           severalJobs.hashCode();
           //System.out.println(severalJobs.toString());
           long EndTime = System.currentTimeMillis();
           long TotalTime = EndTime - StartTime;
           System.out.println("it took"+ TotalTime );
        }
 
 
        public void doWork(String thread)
        {
            log("Starting work on piece '" + pDescripion + "' in thread '" + thread + "'");
 
            long timeOfStart = System.currentTimeMillis();
 
            try
            {
                work();
            }
            catch(Exception e)
            {
                log("An error occured " + e);
                e.printStackTrace(System.err);
            }
 
//            try
//            {
//                Thread.sleep(1000 + new Random().nextInt(1000));
//            }
//            catch(InterruptedException e){}
 
            long timeOfEnd = System.currentTimeMillis();
 
            long totalTime = timeOfEnd - timeOfStart;
 
            log("Finished work on piece '" + pDescripion + "' in thread '" + thread + "' in " + totalTime + "ms");    
        }
    }
 
    private static void log(String text)
    {
        System.out.println(text);
    }
}

Originally Posted by Norm

Does that mean this?
Say there are 6 jobs and 3 threads:
thread one does jobs 1 & 2
thread two does jobs 3 & 4
thread three does jobs 5 & 6

Yes, I need the jobs to be shared or split among the threads

Originally Posted by Norm

Do you see how to compute which jobs are assigned to each thread?
Divide the number of jobs by the number of threads and allocate the jobs from that.

I cant see 'how to compute which jobs are assigned to each thread' Can you direct me to it or give me more hint

Originally Posted by Norm

It depends on how the jobs are defined. See my example in post#36
If the jobs were: 1,2,3,4,5,6 and there were 3 threads
Then the jobs could be assigned this way:
thread one does jobs 1 & 2
thread two does jobs 3 & 4
thread three does jobs 5 & 6

Yes, I ve seen that, how to do it that is my question, How do I split the job among threads

Originally Posted by Norm

How are the "jobs" defined and created? Do you know how many "jobs" will be created? Can you differentiate between one "job" and another?
Is there a first "job" and a second "job" and a third "job" etc?
Can the creation of the first n jobs be given to one thread
and the creation of the second n jobs be given to another thread?

Just as stated earlier, the number of jobs to be processed (generated) are specified in the body of the program, this can be changed after every run inside the program. If I can get a general template, algorithm or whatever on how to split (say N-number of jobs) between (say i number of threads) that will be ok.

Originally Posted by Norm

Give each thread N/i jobs

Ok, at what point do I specify that, because I have to specify not just a number but where each thread starts and ends (isn't it?) or can it just be simply specified as N/i? (thanks in advance)

Originally Posted by Norm

If all the jobs are in an array, then pass each thread indexes into the array for the first and last job it is to process.
For N jobs and T threads
first thread gets: 0, N/T-1
second thread gets: N/T, 2*N/T-1
etc

Can't believe am still stuck on what to do, have tried several methods and it just seems not working