Using ChromeDriver

I am trying to download a page from my own site. I am using webdrivermanager to avoid the problems with changing versions of chrome but I cannot get it to work. This is my code (with some things I tried commented out!):
package com.mycompany.testchromedriver; import io.github.bonigarcia.wdm.WebDriverManager; import org.openqa.selenium.chrome.ChromeDriver; import org.openqa.selenium.chrome.ChromeOptions; /** * * @author NeilB */ public class TestChromeDriver { static ChromeDriver getCD() { // This gets the latest version of Chrome Driver       // WebDriverManager.chromedriver().setup();        WebDriverManager.chromiumdriver().setup();        ChromeOptions option = new ChromeOptions();        option.addArguments("--headless");        option.addArguments("--disable-gpu");        option.addArguments("--window-size=1920,1200");        option.addArguments("--ignore-certificate-errors");              option.addArguments("--silent");        option.addArguments("--remote-debugging-port=9222");     //   option.addArguments("--no-sandbox");      //  option.addArguments("--disable-dev-shm-usage");        option.addArguments("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36", "test-type"); //   option.addArguments("--remote-allow-origins=*");  //      option.addArguments("sec-ch-ua=Google Chrome;v=97");        //    option.addArguments("sec-ch-ua-mobile:?0");        //  option.addArguments("sec-ch-ua-platform:Windows");        //      option.addArguments("disable-features", "UserAgentClientHint");        return new ChromeDriver(option);    }    private static String getPageUsingChrome(String URL) {        ChromeDriver CD = getCD();        CD.get(URL);        String page = CD.getPageSource();        CD.quit();        return page;    }    public static void main(String[] args) {            String readPage = getPageUsingChrome("http://www.neilbarton.co.uk");        System.out.println("page:" + readPage);        System.out.println("Completed");    } }
This is my POM:
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <groupId>com.mycompany</groupId>    <artifactId>TestChromeDriver</artifactId>    <version>1.0-SNAPSHOT</version>    <packaging>jar</packaging>    <dependencies>        <dependency>            <groupId>io.github.bonigarcia</groupId>            <artifactId>webdrivermanager</artifactId>            <version>5.5.3</version>            <type>jar</type>        </dependency>        <dependency>            <groupId>org.seleniumhq.selenium</groupId>            <artifactId>selenium-chrome-driver</artifactId>            <version>4.5.3</version>            <type>jar</type>        </dependency>        <dependency>            <groupId>backport-util-concurrent</groupId>            <artifactId>backport-util-concurrent</artifactId>            <version>3.1</version>            <type>jar</type>        </dependency>    </dependencies>    <properties>        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>        <maven.compiler.source>20</maven.compiler.source>        <maven.compiler.target>20</maven.compiler.target>        <exec.mainClass>com.mycompany.testchromedriver.TestChromeDriver</exec.mainClass>    </properties> </project>
This is the output when I run it:
Starting ChromeDriver 118.0.5993.70 (e52f33f30b91b4ddfad649acddc39ab570473b86-refs/branch-heads/5993@{#1216}) on port 36682
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Exception in thread "main" org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: session not created: Chrome failed to start: was killed.
 (chrome not reachable)
 (The process started from chrome location C:\Program Files (x86)\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

I am running Chrome V 118.0.5993.71 but I don't think the last increment matters, it says it successfully started Chrome! Any thoughts on why it might be crashing?

Have you tried using a more recent version of selenium-chrome-driver?  The current version is 4.14.0.

Thanks Ron,
I changed the Selenium version in the POM to be 4.14.0. It downloaded new files but then gave the same response:
Exception in thread "main" org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: session not created: Chrome failed to start: was killed.
 (chrome not reachable)
 (The process started from chrome location C:\Program Files (x86)\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Host info: host: 'NLB7', ip: '192.168.56.1'
Build info: version: '4.14.0', revision: 'e5a96349ce'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '20.0.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver

When I read about this everything says it's caused by different versions of Chrome and the driver but I'm using webdrivermanager to avoid this. Is this correct?

It seems like the instance of Chrome is failing because you specified two options in the same method call.

After changing this:option.addArguments("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36", "test-type");to this:option.addArguments("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"); option.addArguments("test-type"); it ran without errors (but with a few warnings).
Oct 12, 2023 8:51:47 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch WARNING: Unable to find CDP implementation matching 118 Oct 12, 2023 8:51:47 AM org.openqa.selenium.chromium.ChromiumDriver lambda$new$5 WARNING: Unable to find version of CDP to use for 118.0.5993.71. You may need to include a dependency on a specific version of the CDP using something similar to `org.seleniumhq.selenium:selenium-devtools-v86:4.14.0` where the version ("v86") matches the version of the chromium-based browser you're using and the version number of the artifact is the same as Selenium's. page:<html><head> </head><body><h1>Welcome to Neil Barton's Website</h1> <style> body {font-size: 20px} ...

Thanks Ron,
I've changed the line but it doesn't seem to make a difference, I still get:
Exception in thread "main" org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: session not created: Chrome failed to start: was killed.
 (chrome not reachable)
 (The process started from chrome location C:\Program Files (x86)\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Host info: host: 'NLB7', ip: '192.168.56.1'
Build info: version: '4.14.0', revision: 'e5a96349ce'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '20.0.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--headless, --disable-gpu, --window-size=1920,1200, --ignore-certificate-errors, --silent, --remote-debugging-port=9222, Mozilla/5.0 (Windows NT 10...., test-type], extensions: []}}]}]
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:140)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:96)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:68)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:163)
at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:196)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:171)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:602)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:236)
at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
at org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:108)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:88)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:83)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:72)
at com.mycompany.testchromedriver.TestChromeDriver.getCD(TestChromeDriver.java:35)

I can understand why it might not like multiple arguments on a line but I still can't see what's going wrong. Any other thoughts? I did try it with both the Chrome & Chromium drivers, both failed the same.

Neil - I did my testing using JUnit.  I tried again using ExecJavaMojo and it worked as well.

Looking again at the code, I think that the user-agent and test-type options should probably be prefixed with --, but that shouldn't have caused things to fail.

Here is the code and POM which I ran (I added a dependency for logging and another to remove the browser version-specific warnings):
package com.mcleodnet.chromedriver; import io.github.bonigarcia.wdm.WebDriverManager; import org.openqa.selenium.chrome.ChromeDriver; import org.openqa.selenium.chrome.ChromeOptions; /** * * @author NeilB */ public class TestChromeDriver {    public static final String UA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"            + " AppleWebKit/537.36 (KHTML, like Gecko)"            + " Chrome/118.0.0.0 Safari/537.36";    static ChromeDriver getCD() { // This gets the latest version of Chrome Driver        WebDriverManager.chromiumdriver().setup();        ChromeOptions option = new ChromeOptions()                .addArguments("--headless")                .addArguments("--disable-gpu")                .addArguments("--window-size=1920,1200")                .addArguments("--ignore-certificate-errors")                .addArguments("--silent")                .addArguments("--user-agent=" + UA)                .addArguments("--test-type");        return new ChromeDriver(option);    }    private static String getPageUsingChrome(String URL) {        ChromeDriver CD = getCD();        CD.get(URL);        String page = CD.getPageSource();        CD.quit();        return page;    }    public static void main(String[] args) {        String readPage = getPageUsingChrome("http://www.neilbarton.co.uk");        System.out.println("page:" + readPage);        System.out.println("Completed");    } }<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0"        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <groupId>com.mcleodnet</groupId>    <artifactId>TestChromeDriver</artifactId>    <version>1.0-SNAPSHOT</version>    <dependencies>        <dependency>            <groupId>io.github.bonigarcia</groupId>            <artifactId>webdrivermanager</artifactId>            <version>5.5.3</version>        </dependency>        <dependency>            <groupId>org.seleniumhq.selenium</groupId>            <artifactId>selenium-chrome-driver</artifactId>            <version>4.14.0</version>        </dependency>        <dependency>            <groupId>org.seleniumhq.selenium</groupId>            <artifactId>selenium-devtools-v118</artifactId>            <version>4.14.0</version>        </dependency>        <dependency>            <groupId>org.slf4j</groupId>            <artifactId>slf4j-simple</artifactId>            <version>2.0.9</version>            <scope>runtime</scope>        </dependency>    </dependencies>    <properties>        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>        <maven.compiler.source>17</maven.compiler.source>        <maven.compiler.target>17</maven.compiler.target>        <exec.mainClass>com.mcleodnet.chromedriver.TestChromeDriver</exec.mainClass>        <exec.cleanupDaemonThreads>false</exec.cleanupDaemonThreads>    </properties> </project>mvn -q -DskipTests clean package exec:java       page:<html><head> </head><body><h1>Welcome to Neil Barton's Website</h1> <style> ... </body></html> Completed
My environment:
    Windows: 10.0.19045.2364
    Java: 17.0.7 (Temurin-17.0.7+7)
    Maven: 3.6.3
    Chrome: 118.0.5993.71

Thanks Ron,
I went back to complete basics in the end and got the simple stuff working and then built it up again. Not sure what was wrong but it's all working now :-)

Thanks for your help,
Neil

The error message suggests a problem with starting a new session with ChromeDriver. Check the compatibility between Chrome and ChromeDriver, try running without headless mode, verify system dependencies, and ensure no other Chrome instances are running. Updating both Chrome and ChromeDriver may also help.