Thread: Scraping data from the web

Hi, thanks in advance to whomever can help me.

I am building a java app with Eclipse that will return data to the user depending on what the user types in a text field. The interface is almost done, now I am trying to find the best way to do this.

I understand HTML form is one way to do this, so depending on what the user inputs in the field, the script would then go to the chosen site, input that in the form and then submit. Once found, I coup scrape certain data with Jsoup.

I am uncertain how to store what the user has input, then send it to the site once either “Enter” has been pressed or a button. Once either of those action has happened I need the script to input it in a form on the site.

Any help would be greatly appreciated as I am new to java

Greetings

Thanks for the reply.

It wouldn’t be a browser replacement. The user input would be the ticker symbol (stock market) and input that into one or two websites and collect certain data and return it to the GUI window.

I see what you’re saying. Perhaps I don’t explain it right.

The goal of the application is to allow the user to pull 3 data points from 2 different sites simultaneously. The data will be displayed side by side and the user will be able to compare those numbers at a glance.

Instead of having to open a website, do the research and then take notes and compare. I want to automate this process so that the user can have a little window, input the stock symbol and fetch that data automatically. This is all to save time and effort for traders.

The websites are yahoo finance and finviz.

I believe Yahoo finance has shutdown their API hence why I want to replicate what the browser does.

I'm very new to JAVA in general, so any help will be greatly appreciated.

I'm trying to scrape certain values from a site using Jsoup and I have had success to a certain extent. I am able to scrape a whole row, but I can't seem to figure out how to target one specific cell.

Here is the url: https://finviz.com/quote.ashx?t=nflx
Here is a snapshot of the cell I want, the value.

https://ibb.co/2ZKph4C

Here are the values returned from my code,, which is the whole row.
https://ibb.co/BjK8v8X

here is the code
https://ibb.co/X5swBVZ

Lastly here is the inspection of the element in my web browser. The value seems to be cotained in <b> </b>, how do I access it?

https://ibb.co/xFFNTtw
Thank you

Hello,

As far as I understand you will read data from a website/page.
I had this question also. I have a webserver running on a ESP32 with a BME280 sensor which give temperature, humidity and air pressure.
With the help of the OkHttp librari which example can be find on the internet. https://www.bing.com/videos/search?q...6FORM%3DHDRSC3

With help of string manipulation and unique letter combination I was able to retract the temp, humi, and airpress data.
[code]
package com.example.okhttprequestexample;

import androidx.appcompat.app.AppCompatActivity;
import android.os.Bundle;
import android.widget.TextView;
import java.lang.String;
import java.io.IOException;
import okhttp3.Call;
import okhttp3.Callback;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;




public class MainActivity extends AppCompatActivity {



private TextView mTextViewResult;


@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);

mTextViewResult = findViewById(R.id.text_view_tempb);



OkHttpClient client = new OkHttpClient();

String url = "http://192.168.2.11";

Request request = new Request.Builder()
.url(url)
.build();

client.newCall(request).enqueue(new Callback() {
@Override
public void onFailure(Call call, IOException e) {
e.printStackTrace();
}

@Override
public void onResponse(Call call, Response response) throws IOException {
if (response.isSuccessful()) {
final String myResponse = response.body().string();

MainActivity.this.runOnUiThread(new Runnable() {
@Override
public void run() {
int index_tempb = myResponse.indexOf("Temperature");
// mTextViewResult.setText(myResponse);
String tempb = myResponse.substring(index_tempb-102,index_tempb-97);

System.out.println(myResponse.substring(index_temp b-102,index_tempb-97));
mTextViewResult.setText(myResponse.substring(index _tempb-102,index_tempb-97));


int index_tempi = myResponse.indexOf("Humidity");
String tempi = myResponse.substring(index_tempi-98,index_tempi-93);
//mTextViewResult1.setText(myResponse.substring(inde x_tempi-98,index_tempi-93));
mTextViewResult.setText(myResponse.substring(index _tempi-98,index_tempi-93));

}

});
}


}


});
}

}
[code/]
Succes,

ilioSS