Thread: How do we navigate to a page with the anchor link in Web Crawler program?

Hi,
I am writing a web crawler and I am stuck again with a problem.
How do we navigate to a page with the anchor link:

 <a id="dlstDepartments__ctl2_DepartmentsLink" onclick="return UpdateFrames('5');" 
          href="javascript:__doPostBack('dlstDepartments$_ctl2$DepartmentsLink','')">Beverages</a>
 
 
and here is the Javascript code being called:
 
 
function UpdateFrames(DID)
    {
        top.navbar.location.href="Aisles.aspx?ID="+DID;
        location.href="IconAisles.aspx?ID="+DID;
        return false;
    }
 
 
    function __doPostBack(eventTarget, eventArgument) {
        var theform;
        if (window.navigator.appName.toLowerCase().indexOf("microsoft") > -1) {
            theform = document.frmPageForm;
        }
        else {
            theform = document.forms["frmPageForm"];
        }
        theform.__EVENTTARGET.value = eventTarget.split("$").join(":");
        theform.__EVENTARGUMENT.value = eventArgument;
        theform.submit();
    }

Any help would be highly appreciated
Thanking you

This is where you need to run the javascript I guess or try to figure out a way to parse out the correct url with the information you have there, but if this is a general webcrawler I'd just leave it and focus on proper links.

// Json

Thanks for your response.
The website that I want to crawl has most of the links of this sort. So I can't ignore these links.
How to run JavaScript inside the java program? Any idea? Or how can I achieve the same that is being done in the JavaScript code inside the java program?


Thanking you

Perhaps by using Rhino Javascript engine. Sorry I can't give you any examples though because I've never tried it.

// Json

I have never used and tried it.. Sorry

Which frame do you want, you can always just parse that info out....and then go to it as normal pretending its just a web URL with aspx?... with some variables on the end which you already know.

Chris