Threads are killed randomly in While true loop

[kurt-hardy]

Hi all

Firstly if you need more details, feel free to ask me and I will add them.
I have a problem with two threads dying when there both in a while true statement.
The Main class creates two threads (Start and Stop).
These threads are in a while true as I need them to run forever.
They check times in a database for starting and stopping other processes

When a start date is found, the thread creates another thread to start the process and continues checking for another date.
The stop threads works in the same way.

Iv posted my start thread below.
Any help or ideas will be useful

package startstopthreads;
 
import java.sql.ResultSet;
import java.sql.SQLException;
import java.text.DateFormat;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Date;
 
 
public class StartThread extends Thread {
 
    DbConnection Db;
    ResultSet Rs1 = null;
    Date EventStartDate, EventStopDate, EventSyncDate;
    DateFormat DateFormat = new SimpleDateFormat(GlobalDataStore.DateTimeFormat);
    LoggingToFile MessageLog;
    Date Now = null;
    int Flag = 0;
 
    /**
     * Constructor
     */
    public StartThread() {
        //Creates an instance of the log file for errors / changes
        MessageLog = new LoggingToFile();
    }
 
    /**
     * This method is ran by the Main Thread
     */
    @Override
    public void run() {
        Db = new DbConnection();
        // While true means that this constantly runs
        while (true) {
            // Set up database connection and Gets all the events that are Waiting
            Db.CreateConnection();
            Rs1 = Db.GetEventNames(Flag);
            // Get current time
            Now = new Date();
 
            try {
                while (Rs1.next()) {
                    Event event = new Event(Rs1.getInt("Event_ID"));
                    event.setStartDate(Rs1.getString("Event_StartDate"));
                    event.setThreadFlag(Rs1.getInt("Event_Thread_Flag"));
                    event.setSyncID(Rs1.getInt("Event_Sync_ID"));
 
                    //If the start date is before now, do all the checks
                    EventStartDate = StringToDate(event.getStartDate());
                    if (EventStartDate != null) {
                        if (EventStartDate.before(Now) && event.getThreadFlag() == 0) {
                            //If there is a sync event, edit the stop time
                            if (event.getSyncID() != 0) {
                                event.setSyncDate(Rs1.getString("Sync_StopDate"));
                                EventSyncDate = StringToDate(event.getSyncDate());
                                if (EventSyncDate.after(Now)) {
                                    System.out.println("Found Sync Event");
                                    //Change the stop time to equal the synchronisation
                                    ChangeStopTimeAccordingToSync(event);
                                    //Set up the event object and playout the stream
                                    event.setEventId(Rs1.getInt("Event_ID"));
                                    event.setServerId(Rs1.getInt("PS_ID"));
                                    event.setHostName(Rs1.getString("PS_Host_Name"));
                                    event.setPortId(Rs1.getInt("Event_Ports_ID"));
                                    event.setPortNo(Rs1.getInt("Ports_Number"));
                                    event.setStopDate(Rs1.getString("Event_StopDate"));
                                    event.setInBitrate(Rs1.getInt("Event_InBitrate"));
                                    event.setOutBitrate(Rs1.getInt("Ports_Out_Bitrate"));
                                    event.setStreamName(Rs1.getString("Event_Stream_Name"));
                                    event.setLoops(Rs1.getInt("Event_Loops"));
                                    event.setEventName(Rs1.getString("Event_Name"));
                                    event.setType(Rs1.getInt("Stream_Dektek_Type"));
                                    event.setAllowedPids(Rs1.getString("Event_AllowedPIDs"));
                                    event.setExcludePids(Rs1.getString("Event_ExcludePIDs"));
                                    event.setMapped(Rs1.getString("Event_PIDsMapped"));
                                    PlayStreamOut(event);
                                }
                            } else {
                                //If theres no sync event, just play the stream out
                                event.setStopDate(Rs1.getString("Event_StopDate"));
                                EventStopDate = StringToDate(event.getStopDate());
                                if (EventStopDate.after(Now)) {
                                    System.out.println("Found Event");
                                    //Set up the event object and playout the stream
                                    event.setEventId(Rs1.getInt("Event_ID"));
                                    event.setServerId(Rs1.getInt("PS_ID"));
                                    event.setHostName(Rs1.getString("PS_Host_Name"));
                                    event.setPortId(Rs1.getInt("Event_Ports_ID"));
                                    event.setPortNo(Rs1.getInt("Ports_Number"));
                                    event.setStopDate(Rs1.getString("Event_StopDate"));
                                    event.setInBitrate(Rs1.getInt("Event_InBitrate"));
                                    event.setOutBitrate(Rs1.getInt("Ports_Out_Bitrate"));
                                    event.setStreamName(Rs1.getString("Event_Stream_Name"));
                                    event.setLoops(Rs1.getInt("Event_Loops"));
                                    event.setEventName(Rs1.getString("Event_Name"));
                                    event.setType(Rs1.getInt("Stream_Dektek_Type"));
                                    event.setAllowedPids(Rs1.getString("Event_AllowedPIDs"));
                                    event.setExcludePids(Rs1.getString("Event_ExcludePIDs"));
                                    event.setMapped(Rs1.getString("Event_PIDsMapped"));
                                    PlayStreamOut(event);
                                }
                            }
                        }
                    }
                }
                //Close the connection and sleep the thread for half a second.
                Db.Close();
                this.sleep(500);
                this.yield();
            } catch (SQLException ex) {
                System.out.println("Exception in getting event names - " + ex);
                MessageLog.write("Exception in StartThread class. Method - Run (Thread). SQLException - " + ex);
            } catch (InterruptedException ex) {
                System.out.println("Exception in sleeping Start Thread - " + ex);
            }
        }
    }
 
    /**
     * Gets the length of the Sync event and sets the stop time to equal now + length
     * @param ev - the event object passed in
     */
    private void ChangeStopTimeAccordingToSync(Event ev) {
        //Get the length of the Sync event so we can change the stop time
        int Length = Db.GetSyncLength(ev.getSyncID());
        //Gets whether the length is in hours, minutes or seconds
        int TimeIn = 0;
        switch (Db.GetSyncFlag(ev.getSyncID())) {
            case 1:
                TimeIn = Calendar.HOUR;
                break;
            case 2:
                TimeIn = Calendar.MINUTE;
                break;
            case 3:
                TimeIn = Calendar.SECOND;
                break;
        }
        //Create calendar object and add the length onto now
        Calendar NewDate = Calendar.getInstance();
        NewDate.add(TimeIn, Length);
        //Change stop time to equal Now plus 'length' in the future
        Db.StopEvent(ev.getEventId(), DateFormat.format(NewDate.getTime()));
    }
 
    /**
     * Converts the string passed in to a date object and then to a calendar object
     * @param s - the string to convert
     * @return A calendar object containing the date of the string
     */
    private Date StringToDate(String s) {
        Date date = null;
        try {
            //try parsing once
            date = DateFormat.parse(s);
        } catch (ParseException ex) {
            try {
                //if fail, try again with seconds
                date = DateFormat.parse(s + ":00");
            } catch (ParseException ex2) {
                System.out.println("Exception in parsing string to Date in Start Thread - " + ex);
                MessageLog.write("Exception in StartThread class. Method - StringToDate(Thread). ParseException - " + ex);
            }
        }
        return date;
    }
 
    /**
     * Checks the Stream type is not 0, if so set this to a normal transport stream ( type 1 )
     * @param evnt - the event object of which to create a thread to playout
     */
    private void PlayStreamOut(Event evnt) {
        //If stream type == null, set type to 1
        if (evnt.getType() == 0) {
            evnt.setType(1);
        }
        //Sets the thread flag to 1 so only one flag is created
        Db.SetThreadFlag(evnt.getEventId(), 1);
 
        //Starts the thread and passes the event object to it
        StartStreamThread Start = new StartStreamThread(evnt);
        Thread StartStream = new Thread(Start);
        StartStream.start();
    }
}

Also to help any readers, the Event object just holds variables, getters and setters
The MessageLog logs any changes and exceptions

Thanks
Kurt

[Sean4u]

How do you know the Threads have been 'killed'? I would complete my catch clauses with a catch-all 'catch (Exception e){ e.printStackTrace(); }, just in case some other (than the two you do have catch clauses for) Exception is thrown.

[kurt-hardy]

Im running this under linux and i can see that its killed using 'ps aux' and grepping java.
In the log file, which prints out debug and exceptions, there are no exceptions thrown and the debug all looks normal.

But i will try that and see if it catches anymore exceptions.
Thanks
Kurt

[Sean4u]

ps aux on my pc doesn't give me java threads. Try ps -eLf (from the manpages) that shows over 30 threads (who knew) for a Java application server I'm currently working on, though it doesn't tell me what they're doing. You need jconsole for that (comes with JDK now - I think) or visualvm.

[kurt-hardy]

That shows how many threads are running for each process but when I manually kill the Pid for the java process, the threads relating to the thread class are removed.
Im running this on command line (using a jar file) so when i use grep on java, the command I ran is shown.
This is what is being killed randomly, killing both threads.
It usually takes about a day for this to be killed so Im still testing your idea of putting a catch (Exception e){...}

Thanks
Kurt

[Sean4u]

when I manually kill the Pid for the java process, the threads relating to the thread class are removed

What did you expect to happen? Java threads are not separate processes. What you kill when you kill the 'parent' is the JVM - everything dies.

It usually takes about a day for this to be killed

What might be happening is that you're running out of memory. The MySQL client library is great for this. You're creating a new Connection and creating a new RecordSet every half a second and they might be quite weighty objects. The MySQL stuff requires explicit disposal (with .close()), it doesn't get garbage collected. It might be worth you monitoring memory usage with top, or directly for the running JVM with jconsole or visualvm.

[kurt-hardy]

when I manually kill the Pid for the java process, the threads relating to the thread class are removed

What did you expect to happen? Java threads are not separate processes. What you kill when you kill the 'parent' is the JVM - everything dies.

Sorry a miss understanding on my part. I know the JVM is killed but that is what is happening somewhere in my code.[/QUOTE]

It usually takes about a day for this to be killed

What might be happening is that you're running out of memory. The MySQL client library is great for this. You're creating a new Connection and creating a new RecordSet every half a second and they might be quite weighty objects. The MySQL stuff requires explicit disposal (with .close()), it doesn't get garbage collected. It might be worth you monitoring memory usage with top, or directly for the running JVM with jconsole or visualvm.

I have a close method which is called every half a second aswell. (i'll post it below) which closes the relevant mysql variables. Is this method ok?

I started the program on Tuesday and came in this morning and it wasn't running and there were no debug so putting catch(Exception e){..} didnt work (Im going to start using that anyway as it seems like a good idea _Thanks)

Also Im going to monitor the program today using jconsole. Could you advise on what to look for within the jconsole gui?
Is there a maximum number of threads/CPU/heap that the JVM is allowed to use?

Thanks for your help
Kurt

   public void Close() {
        try {
            if (Rs1 != null) {
                Rs1.close();
            }
            if (stmt != null) {
                stmt.close();
            }
            if (conn != null) {
                conn.close();
            }
        } catch (SQLException se) {
            System.out.println("Closing Exception - " + se.getMessage());
        }catch (Exception e){
            System.out.println("Unknown Exception - "+e.getMessage());
        }
    }

[Sean4u]

I'm not sure jconsole will catch all memory leaks - perhaps someone more familiar with memory management might jump in - if the memory leak is in database connector code accessed via a native code interface. Top (something like 'top -b | grep java') might do a better job. Checking on the code regularly with visualvm (you can take regular heapdumps with jmap) and then look at the heapdumps with visualvm later, might help you spot rising numbers of stub objects.

Your close code should be in a finally block somewhere close to the place where you use the database objects, and should be in separate try blocks to avoid the first close throwing an exception and the remaining closes are never executed. Something like:

while (/* whatever */)
  try
  {
    /* attempt db access */
  }
  catch (Exception e)
  {
    e.printStackTrace();
  }
  finally
  {
    try { rs.close() }
    catch (Exception e){/* don't really care */}
    try { stmt.close() }
    catch (Exception e){/* don't really care */}
    try { conn.close() }
    catch (Exception e){/* don't really care */}
  }

As I recall, that's very similar to a recipe put up on mysql.com somewhere precisely for proper disposal of MySQL connector objects, by Mark somebody who posts a lot there. If that is what's biting you and you go looking around the mysql site, I'm sure you'll find it.