Thread: Problems reading a pgm-file (pixel gray map).

Hi.

First merry christmas to everyone

I've been struggling with some java code to read a pgm-file. I've looked searched a lot on the internet and looked at several different examples and my code is almost working, but i still experience some weird errors.

The PGM file format is a very simple one and the full specification can be seen here.

So i've written a method that generates and saves a pgm file in the root folder. To test it i generated an array containing image data for an image with size 24*16 and greytone values from 0 to 15. With each line of pixels i increment the greytone-value so that i get 16 lines with different greytone values. So the imagedata looks like this:

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7
8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8
9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9
10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11
12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12
13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13 13
14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15

When i open the file in an image-program that can display pgm-files it loads just fine and the image has no flaws or anything. However i get problems when trying to read the file from the root-folder back into an array of ints containing all the different pixel-values:

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6
6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7
7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9
9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 10 10 10 10 10 10 10 10 10
10 10 10 10 10 10 10 10 10 10 10 10 10 10 11 11 11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11 11 11 11 11 11 12 12 12 12 12 12 12 12 12 12 12
12 12 12 12 12 12 12 12 12 12 12 12 13 13 13 13 13 13 13 13 13 13 13 13
13 13 13 13 13 13 13 13 13 13 13 14 14 14 14 14 14 14 14 14 14 14 14 14
14 14 14 14 14 14 14 14 14 14 15 15 15 15 15 15 15 15 15 15 15 15 15 15
15 15 15 15 15 15 15 15 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

I've had different variations of errors, but so far this is the least bugged read i've been able to produce. The first line is fine. i get 24 0's which is what is supposed to be in the first line. But then it's like it skips one byte of data in every line of pixels and in the end it inserts 0's in for all those bytes that it missed earlier. But why oh why?? I know that the bytes are there! Here is my code:

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.DataInputStream;
import java.io.EOFException;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.InputStreamReader;
import java.util.Scanner;
 
public class testpgm {
 
	public testpgm()
	{	
	}
 
	int width;
	int height;
	int[][] imgData;
	//ByteBuffer buffer;
	public void writePGM(int[] img,int w, int h, int r, String s)
	{
		String newline = new String("\n");
		if(img.length==w*h)
		{
		try{
			  FileWriter fstream = new FileWriter(s+".pgm");
			  BufferedWriter out = new BufferedWriter(fstream);
			  out.write("P5"+newline);
			  out.write(Integer.toString(w)+" "+Integer.toString(h)+newline);
			  out.write(Integer.toString(r-1)+newline);
			  int i = 0;
			  while(i<w*h)
			  {
				  out.write((byte)img[i]);
				  i++;
			  }
			  out.close();
			  }catch (Exception e){//Catch exception if any
			  System.err.println("Error: " + e.getMessage());
			  }
		}
		else
		{
			System.out.println("Error");
		}
	}
 
	public static void main(String[] args) throws Exception {
 
		testpgm image = new testpgm();
		int[] img = new int[24*16];
		int w=0;
		int h=0;
 
		while(h<16)
		{
			if(w<24)
			{
				img[w+(h*24)]=h;
				w++;
			}
			else
			{
				h++;
				w=0;
			}
		}
		image.writePGM(img, 24, 16, 16, "billede");
 
		try {
            BufferedReader d = 
					new BufferedReader(
							new InputStreamReader(
									new DataInputStream(
											new FileInputStream("billede.pgm"))));
            String magic = d.readLine();    // first line contains P2 or P5
            String line = d.readLine();     // second line contains height and width
            while (line.startsWith("#")) {
                line = d.readLine();
            }
            Scanner s = new Scanner(line);
            int width = s.nextInt();
            int height = s.nextInt();
            line = d.readLine();// third line contains maxVal
            s = new Scanner(line);
            int maxVal = s.nextInt();
            int[][] im = new int[width][height];
            System.out.println(maxVal);
            int x=0;
            int y=0;
            int count = 0;
            int b = 0;            
            try {
                while (count < height*width) {
                    b = d.read();
                   //System.out.println(b);
                    if ( b < 0 || b>=maxVal) 
                        break ;
                        if ( "P5".equals(magic) ) { //Magic number p5, binary format
                        	if(x<width)
                        	{
                        		im[x][y] = b;
                        		//System.out.println("x: "+x+" y: "+y+" tone: "+b);
                                x++;
                        	}else
                        	{
                        		y++;
                        		x=0;
                        	}          
                        }
                        else {  // Magic number p2, ASCII format
                            im[count %width][count /width] = b ;
                            count++;
                        }
                }
            } catch (EOFException eof) {
                eof.printStackTrace(System.out) ;
            }
            System.out.println("Height=" + height);
            System.out.println("Width=" + height);
            System.out.println("Required elements=" + (height * width));
            System.out.println("Obtained elements=" + count);
            System.out.println("value=" + im[22][14]);
            x = 0;
            y = 0;   
            while(y < height)
            {
            	if(x<width)
            	{
            		System.out.print(im[x][y]+" ");
            		x++;
            	}
            	else
            	{
            		System.out.println(" ");
            		y++;
            		x=0;
            	}
            }
        }
        catch(Throwable t) {
            t.printStackTrace(System.err) ;
            return ;
        }
    }
}

I will only be working with pgm-files with magic number p5. I would be very thankful if anyone could help me figure out what is going wrong in this code.

Originally Posted by Norm

Can you print out the contents of the array and add some comments that show the problem?

I'm not sure what you mean? I did print the contents of the array both in my previous post and in my code. This is how it looks in my console-window:

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 This line is the only one that is fine. 24 0's is correct.
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 In this line it should find 24 1's just like it found 24 0's, but it only finds 23.
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 This problem seems to consistently continue...
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 It seems to miss a byte everytime the pixelvalue changes.
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 Thus it only finds 23 bytes of each pixel-value, but i know that there are 24.
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 The image-file loads correctly in GIMP (a program that can open pgm-files).
6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 I can see that all the data is correct.
7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 So there is no problem in the part of the code that generates the pgm-file
8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 The problem occurs in the part of the code that attempts to read the pgm-file.
9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 10 10 10 10 10 10 10 10 10
10 10 10 10 10 10 10 10 10 10 10 10 10 10 11 11 11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11 11 11 11 11 11 12 12 12 12 12 12 12 12 12 12 12
12 12 12 12 12 12 12 12 12 12 12 12 13 13 13 13 13 13 13 13 13 13 13 13
13 13 13 13 13 13 13 13 13 13 13 14 14 14 14 14 14 14 14 14 14 14 14 14
14 14 14 14 14 14 14 14 14 14 15 15 15 15 15 15 15 15 15 15 15 15 15 15
15 15 15 15 15 15 15 15 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 So far, the pixelvalue has changed 15 times and the code seems to have lost 15 byte, and for some reason it has printed 15 0's after the last pixel valued 15. So the problem is that for some reason, a byte goes missing every time the pixel-value in the image changes.

This is the part of the code where the array with image-data is generated:

testpgm image = new testpgm();
int[] img = new int[24*16];
		int w=0;
		int h=0;
 
		while(h<16)
		{
			if(w<24)
			{
				img[w+(h*24)]=h;
				w++;
			}
			else
			{
				h++;
				w=0;
			}
		}
image.writePGM(img, 24, 16, 16, "billede");

This is the method that can generate PGM-files:

public void writePGM(int[] img,int w, int h, int r, String s)
	{
		String newline = new String("\n");
		if(img.length==w*h)
		{
		try{
			  FileWriter fstream = new FileWriter(s+".pgm");
			  BufferedWriter out = new BufferedWriter(fstream);
			  out.write("P5"+newline);
			  out.write(Integer.toString(w)+" "+Integer.toString(h)+newline);
			  out.write(Integer.toString(r-1)+newline);
			  int i = 0;
			  while(i<w*h)
			  {
				  out.write((byte)img[i]);
				  i++;
			  }
			  out.close();
			  }catch (Exception e){//Catch exception if any
			  System.err.println("Error: " + e.getMessage());
			  }
		}
		else
		{
			System.out.println("Error");
		}
	}

There are no problems in the code thus far. The code above works fine as intended.
Problem occurs in this piece of code that attempts to read that pgm-file that got saved to the root folder in the previous code:

try {
            BufferedReader d = 
					new BufferedReader(
							new InputStreamReader(
									new DataInputStream(
											new FileInputStream("billede.pgm"))));//pgm-file from previous code.
            String magic = d.readLine();    // first line contains P2 or P5
            String line = d.readLine();     
            while (line.startsWith("#")) {
                line = d.readLine();
            }
            Scanner s = new Scanner(line);//Second line contains width and height coded as ascii seperated by whitespace
            int width = s.nextInt();
            int height = s.nextInt();
            line = d.readLine();// third line contains maxVal
            s = new Scanner(line);
            int maxVal = s.nextInt();
            int[][] im = new int[width][height];
            System.out.println(maxVal);
            int x=0;
            int y=0;
            int count = 0;
            int b = 0;            
            try {
                while (count < height*width) {// I believe the problem occurs in this while-loop, but don't know why it happens.
                    b = d.read();
                   //System.out.println(b);
                    if ( b < 0 || b>=maxVal) 
                        break ;
                        if ( "P5".equals(magic) ) { //Magic number P5, binary format
//specifically problem occurs somewhere arround here since i'm dealing with pgm-files with P5 as magic-number.
                        	if(x<width)
                        	{
                        		im[x][y] = b;
                        		//System.out.println("x: "+x+" y: "+y+" tone: "+b);
                                x++;
                        	}else
                        	{
                        		y++;
                        		x=0;
                        	}          
                        }
                        else {  // Magic number p2, ASCII format
                            im[count %width][count /width] = b ;
                            count++;
                        }
                }
            } catch (EOFException eof) {
                eof.printStackTrace(System.out) ;
            }
            System.out.println("Height=" + height);
            System.out.println("Width=" + height);
            System.out.println("Required elements=" + (height * width));
            System.out.println("Obtained elements=" + count);
            System.out.println("value=" + im[22][14]);
            x = 0;
            y = 0;   
            while(y < height)
            {
            	if(x<width)
            	{
            		System.out.print(im[x][y]+" ");
            		x++;
            	}
            	else
            	{
            		System.out.println(" ");
            		y++;
            		x=0;
            	}
            }
        }
        catch(Throwable t) {
            t.printStackTrace(System.err) ;
            return ;
        }

Hi Norm. Thanks a lot for your reply, it is appreciated So i should put that little snippet at the end of my code right?? I'll try that and see if i can find out what goes wrong I may post again if i get stuck with it

Merry Christmas.

Got it working now. Also found out that i accidentally pasted the wrong code above, i had so many different projects open trying different versions of this code, lol. I'm still not sure what went wrong, but found something that looked weird and deleted it and everything works now. I'm probably quite messy at times when i code, especially when i'm trying to do something i haven't done before, i should train to be more structured so i find my mistakes quicker. I'm still only beginning to learn how to program, but it's incredible the amount of time i can spend at times on silly little mistakes.

Thanks a lot for taking your time to reply

--- Update ---

Okay now i'm getting a different problem when i move on to the bigger pgm-file that i need to use for this project. It's not a big problem and i can probably work my way around it, but it's weird none the less. The image is of a map, which is meant to simulate an environment for a robot, so i can build navigation algorithms. There are 3 different tones in the image, 0 (black), maxvalue which is 255 (white) and some tone in between. I get the right number of elements, but i get weird values of 711 and 8225. I can easily make a method that cleans up this array and replaces those weird values, but i'd like to understand why it happens. When i read the information on the read() method i use on the BufferedReader object it says that it reads a single character and returns an int. I'm guessing that this is an ascii-character which i assume is essentially just a byte. Am i wrong in this?? Here's the snippet that reads the file as it looks now:

try {
            while (count < height*width) {
                b = d.read();
                if ( b < 0 )
                {
                    break ;
                }
                else {
                    if ( "P5".equals(magic) ) // Binary format
                    { 
                        im[count %width][count /width] = b;
                        count++;
                    }
                    else // ASCII format
                    {  
                        im[count %width][count /width] = b;
                        count++;
                    }
                }
            }
        } catch (EOFException eof) {
            eof.printStackTrace(System.out) ;
        }

Also is there something similar to the BufferedReader class which has a method that returns raw bytes (or their values as ints) in a similar fashion as read returns characters as ints?

No i have not. I'm not familiar with HexEditors?? How do i do this? I just tried some different ways of converting b in the above snippet to a byte and then converting it to an int when i load it in my map. Also tried changing to a byte-array. The values are different then the first values but still weird.

Just took a small look at the summary of FileInputStream. However if i use this, then i'm not sure what to do about the information in beginning of file, according to the file format specification. This information in beginning is coded as ascii-characters and stored in the first 3 lines. There are no readLine() method for FileInputStream, or method that reads as ascii, so how would i go about reading/discarding the first 3 lines and finding the right spot where the byte-array with image-data starts?? I know that when writing ascii characters '\n' means new line, is that analogous to some specific byte-value that i can look for in the beginning of the file?

Yes i print out all the values of b in the end of my main. All the white space has value 255, but is read as 711. The extra grey-tone in the image has value 150 (i think) and is read as 8225. All the black pixels are correctly read as 0.

ways of converting b in the above snippet to a byte...
What does that mean? An int is made of 4 bytes. You get each byte by doing an AND to remove the other bytes and/or shift of the int.

Well, yeah that was what i meant. I did something like (byte)((b>>8)&0xFF)... and then i did (int)(b) when loading into im[][], which kind of seemed to be an odd way of going about it... didn't work either...

So i loaded in Hex Fiend, and i get what i expect. Mostly F's, all the white space is value 255 which is FF, some 0's which is all the black space, so the file looks fine. Nice tool that hex editor for looking at files when doing stuff like this btw. Now i know that the code is somehow reading it wrong. It must be something with the way that read() method in BufferedReader returns that int. Looking closer at read() it says that it reads a single character and returns an int between 0 and 65535 suggesting that it is actually looking at 2 bytes at a time. If so, then how is it that i obtain the right number of elements for the array?? As soon as i remove some of the bytes with the hex editor i get an inconcistent number of elements for the array when running the program.

I've found out that the problem only occurs when there are pixel values in the image greater than 127. So it must be the MSB of each byte that somehow gets misinterpreted.