Open a URL and read it line by line (Works in Eclipse but not from Command Line)

**rosco544** · September 16th, 2011, 03:56 PM

Hi, I am trying to use this code which works fine in Eclipse but when I try to execute the program from the command line it just hangs forever and can never accesses the URL.

I am trying to run this program from my home computer on a localhost and I have no idea why this isn't working. I read online and some articles suggested using a proxy but I am not behind a proxy or anything. And again this code works fine when I run it in Eclipse so I guess Eclipse is auto-configuring settings to get this code to work.

URL link = new URL("http://www.yahoo.com");
System.out.println("got to this line 1");
BufferedReader in = new BufferedReader(new InputStreamReader(link.openStream()));
System.out.println("got to this line 2");
//InputStream in = link.openStream();
String inputLine = "";
int count = 0;
while ((inputLine = in.readLine()) != null)
{
System.out.println("line: " + inputLine);
site = site + "\n" + inputLine;
}
in.close();
System.out.println("got to this line 3");

It gets to the print statement inside of the while loop then all of a sudden the connection gets reset and it stops printing the file. It always stops at the same place.

**copeg** · September 16th, 2011, 04:31 PM

'it just hangs forever

Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

This thread has been cross posted here:

http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

Although cross posting is allowed, for everyone's benefit, please read:

Java Programming Forums Cross Posting Rules

The Problems With Cross Posting

**Sean4u** · September 16th, 2011, 04:36 PM

Post your full source, cut-down so that it demonstrates the problem, but can still be copy-paste-compile-run if someone feels like it. Change the body of your while loop to print inputLine so you can see progress of the stream. You know you can specify a file: URL and test your code against a file on your PC? Like:

URL link = new URL("file:///home/rosco/dearsanta.text");

**rosco544** · September 16th, 2011, 04:37 PM

Originally Posted by copeg

Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

This thread has been cross posted here:

http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

Although cross posting is allowed, for everyone's benefit, please read:

Java Programming Forums Cross Posting Rules

The Problems With Cross Posting

Sorry about the cross-posting I didn't realize the two forums had the same viewers. I edited the original post to include the full source and if a solution is found I will post it and let everyone know in both threads.

**rosco544** · September 16th, 2011, 04:42 PM

Originally Posted by copeg

Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

This thread has been cross posted here:

http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

Although cross posting is allowed, for everyone's benefit, please read:

Java Programming Forums Cross Posting Rules

The Problems With Cross Posting

I edited the code to look like this in my program and re-ran it from the command line.

URL link = new URL("http://www.yahoo.com");
System.out.println("got to this line 1");
BufferedReader in = new BufferedReader(new InputStreamReader(link.openStream()));
System.out.println("got to this line 2");
//InputStream in = link.openStream();
String inputLine = "";
int count = 0;
while ((inputLine = in.readLine()) != null)
{
site = site + "\n" + inputLine;
}
in.close();
System.out.println("got to this line 3");

It gets to "got to this line 2" but not "got to this line 3".

**rosco544** · September 16th, 2011, 04:47 PM

Originally Posted by copeg

Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

This thread has been cross posted here:

http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

Although cross posting is allowed, for everyone's benefit, please read:

Java Programming Forums Cross Posting Rules

The Problems With Cross Posting

I edited the code to look like this in my program and re-ran it from the command line.

URL link = new URL("http://www.yahoo.com");
System.out.println("got to this line 1");
BufferedReader in = new BufferedReader(new InputStreamReader(link.openStream()));
System.out.println("got to this line 2");
//InputStream in = link.openStream();
String inputLine = "";
int count = 0;
while ((inputLine = in.readLine()) != null)
{
System.out.println("line: " + inputLine);
site = site + "\n" + inputLine;
}
in.close();
System.out.println("got to this line 3");

It starts printing the website but never gets to "got to this line 3".

**Sean4u** · September 16th, 2011, 04:47 PM

There's nothing wrong with the fragment of code you've posted. Create a small, compilable... (Kevin? I must make one of them fancy footnote things with all the handy links in). Write the smallest possible java program with a main method so that it can be run from the command line that demonstrates the problem. While you're doing that, you'll probably spot for yourself where you've gone wrong.

**rosco544** · September 16th, 2011, 04:53 PM

Originally Posted by Sean4u

There's nothing wrong with the fragment of code you've posted. Create a small, compilable... (Kevin? I must make one of them fancy footnote things with all the handy links in). Write the smallest possible java program with a main method so that it can be run from the command line that demonstrates the problem. While you're doing that, you'll probably spot for yourself where you've gone wrong.

Well I don't think I've done anything wrong because the entire project/program works flawlessly in Eclipse. I think it has something do with the Proxy settings that the JVM uses. I just don't know anything about proxies to be able to configure that.

**Sean4u** · September 16th, 2011, 05:08 PM

the entire project/program works flawlessly in Eclipse

Great - all you need to do is to distribute your application with the Eclipse Virtual Machine.

You'd know if your JVM was using proxy settings - you'd have specified them on the command line. As far as I know, the JVM won't silently go looking for a web proxy if you don't tell it to. Are you behind a web proxy on your network?

**rosco544** · September 16th, 2011, 05:11 PM

No I'm just on a standard home network. I do have an apache server running on the localhost if that matters, so I can use mysql and php and stuff. But I don't think that has anything to do with it.

And is it possible to distribute a console application using a jar. I thought jar files couldn't start the console in Windows? The reason why I want to be able to run it from the command is because I want to then create a script to be able to run this automatically at intervals, and eventually I want it to be able to run on a Web server at intervals.

**rosco544** · September 16th, 2011, 07:49 PM

Finally solved it with this code found here: Java HttpURLConnection (how to open and read a url with Java) | devdaily.com

public String websiteToString(String desiredUrl) throws Exception
{
URL url = null;
BufferedReader reader = null;
StringBuilder stringBuilder;

try
{
// create the HttpURLConnection
url = new URL(desiredUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();

// just want to do an HTTP GET here
connection.setRequestMethod("GET");

// uncomment this if you want to write output to this url
//connection.setDoOutput(true);

// give it 15 seconds to respond
connection.setReadTimeout(15*1000);
connection.connect();

// read the output from the server
reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
stringBuilder = new StringBuilder();

String line = null;
while ((line = reader.readLine()) != null)
{
stringBuilder.append(line + "\n");
}

String site = stringBuilder.toString();
site = site.replaceAll("<html([^<]*)>", "<html>"); // replace all <html xml:lang="en" lang="en" xmlns

ml="xml"> with <html>
// this is important because xpath can't parse an expression with <html> tags with junk in it

return site;
}
catch (Exception e)
{
e.printStackTrace();
throw e;
}
finally
{
// close the reader; this can throw an exception too, so
// wrap it in another try/catch block.
if (reader != null)
{
try
{
reader.close();
}
catch (IOException ioe)
{
ioe.printStackTrace();
}
}
}
}

Thread: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

LinkBack

Thread Tools

Display

Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Related threads:

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

Similar Threads

How to Send command line arguments in Eclipse?

Reading a file line by line using the Scanner class

Reading a file line by line using the Scanner class

How to Read a file line by line using BufferedReader?

How to Read a file line by line using BufferedReader?

Tags for this Thread