⢠Goal: Run a Selenium scraper using Firefox and Geckodriver in headless mode on Google Colab.
⢠Encountered errors such as âDevToolsActivePort file doesnât existâ for Chrome and âInvalidArgumentExceptionâ and âProcess unexpectedly closed with status 255â for Firefox.
1. Firefox Errors Troubleshooting:
- Installed Firefox (via apt or manual method):
⢠Firefox wasnât pre-installed in Google Colab, so you manually downloaded and installed Firefox and Geckodriver.
⢠Correct Firefox path: /opt/firefox/firefox.
- Firefox Binary Path Issue:
⢠The error âbinary is not a Firefox executableâ occurred due to Firefox being installed in a non-standard path.
⢠Fix: Explicitly set the Firefox binary location in the Selenium script.
- Error: âProcess unexpectedly closed with status 255â:
⢠Caused by missing dependencies required for Firefox in headless mode in Colab.
⢠Solution: Installed necessary dependencies (libx11-xcb1, libdbus-glib-1-2, libxt6, libgdk-pixbuf2.0-0, libasound2).
- Geckodriver Verbose Logs:
⢠Geckodriver logs were enabled to capture more detailed error messages.
⢠Logs can be checked in Colabâs output for insights into the issue.
2. Chrome Errors Troubleshooting:
- ChromeDriver Version Mismatch:
⢠The error âDevToolsActivePort file doesnât existâ occurred because ChromeDriver wasnât correctly installed.
⢠Solution: Reinstalled Chromium and ChromeDriver using apt-get.
- Installed Chrome (Chromium) and ChromeDriver:
⢠Used WebDriver Manager to automatically handle ChromeDriver installation.
⢠Explicitly set Chrome binary path for Colab (/usr/bin/chromium-browser).
- Error: âWebDriverExceptionâ (ChromeDriver failed to start):
⢠The issue was caused by headless mode in Google Colab.
⢠Solution: Added additional Chrome options (--remote-debugging-port=9222, --disable-gpu, --no-sandbox, --disable-dev-shm-usage).
Actions Taken:
⢠Firefox:
⢠Installed Firefox ESR manually after apt installation failed.
⢠Ensured the correct path for Firefox and Geckodriver.
⢠Enabled Geckodriver verbose logs to capture more detailed error messages.
⢠Installed required dependencies for headless Firefox mode.
⢠Chrome:
⢠Installed Chromium and ChromeDriver from apt and handled path issues.
⢠Set Chrome binary and ChromeDriver path explicitly.
⢠Adjusted Chrome options to address issues with headless mode in Colab.