ScreenAI is a Python-based screen automation and navigation system built on top of Microsoft's OmniParser (a vision-language model for GUI understanding). It takes screenshots, parses UI elements using computer vision and OCR, and then programmatically navigates websites - specifically job board sites LinkedIn and Hirify. The repository bundles the entire OmniParser project as a subdirectory and adds custom navigators, screen parsers, and vacancy estimators on top.

The problem it solves is automated job vacancy collection and evaluation. The Navigators visit URLs from CSV files (linkedin_urls.csv, hirify_urls.csv), capture screen state, parse job listing content out of MHTML files, and use LLMs (via OpenAI, Anthropic, Groq, or local llama.cpp) to estimate/evaluate vacancies. Essentially it's a job-scraping and screening bot that operates through the actual browser UI rather than APIs.

The intended user appears to be the developer themselves - likely someone doing automated job searching or building a job-aggregation dataset. There is no README, no license, no packaging, and main.py contains commented-out experimental code paths, indicating this is a personal research/tooling project rather than a product for external users.

How it works: main.py loads configuration, instantiates a Navigator (LinkedIn or Hirify), and calls run_on_urls(). Navigators use PyAutoGUI to control mouse/keyboard, screeninfo/screenshots to capture the display, and OmniParser (YOLO + BLIP2/Florence-based icon detection plus EasyOCR/PaddleOCR) to identify clickable elements. Captured pages are saved as MHTML, then BaseVacancyEstimator parses them with BeautifulSoup and feeds cleaned text to an LLM for scoring. CUDA/GPU is used for the vision models.

Notable observations: the requirements.txt is enormous (~180 pinned packages, some with future-dated versions like certifi==2026.6.17, pandas==3.0.3, torch 2.11.0+cu128 - versions that don't exist as of this analysis, suggesting the file was generated from a bleeding-edge or possibly fabricated environment). The bundled OmniParser subproject includes a Windows 11 VM Docker setup with an 'execute_anything' Flask endpoint that runs arbitrary shell commands with no auth - dangerous but consistent with OmniParser's upstream research code. No license file is present in the root project.