How to find broken links in Selenium WebDriver

Broken links or dead links are those links that are displayed as a link on the webpage but when we click on them nothing happens. There is no action defined on click. Either because there is no destination defined for the link or due to the destination giving no response.

What is a broken link and how to find them?

In order to find dead links in Selenium, we use two concepts.

1. href attribute in a link (<a> tag) defines the destination but if that is empty or not defined, the tag has nowhere to redirect the user.
2. If we can hit an endpoint, it returns 200 OK and if it is not reachable, it responds with 400 code.

Now taking forward these known concepts, we will write a program using the class HttpURLConnection.

First, we will find all the links on the webpage using the <a> tag. Next, we will run a for loop over all the <a> tags and find if href attributes are defined for them. If yes, we will see if they respond a 200 OK status code using HttpURLConnection provided method.

Example code to find broken links in Selenium

/**
 * This method returns number of broken links
 * @param driver- webdriver driver instance
 * @param pageUrl- Url to be checked for broken links 
 * */   
    
public static int findBrokenLinks(WebDriver driver, String pageUrl) {

    String url = "";
    HttpURLConnection huc = null;
    int respCode = 200;
    int countBrokenLink = 0;

    driver.get(pageUrl);

    List < WebElement > links = driver.findElements(By.tagName("a"));
    Iterator < WebElement > it = links.iterator();
    while (it.hasNext()) {
        url = it.next().getAttribute("href");
        System.out.println(url);
        if (url == null || url.isEmpty()) {
            System.out.println("URL is either not configured for anchor tag or it is empty");
            continue;
        }

        try {
            huc = (HttpURLConnection)(new URL(url).openConnection());
            huc.setRequestMethod("HEAD");
            huc.connect();
            respCode = huc.getResponseCode();
            if (respCode >= 400) {
                System.out.println(url + " is a broken link");
                countBrokenLink = countBrokenLink + 1;
            } else {
                System.out.println(url + " is a valid link");
            }
        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    return countBrokenLink;
 }