Development Environment Setup

The following files were used as context for generating this wiki page:

Purpose and Scope

This guide provides step-by-step instructions for setting up a local development environment for WSHawk. It covers prerequisites, repository cloning, dependency installation, and verification procedures necessary to begin contributing code.

For information about the project's architecture and module organization, see Project Structure and Architecture. For guidance on implementing specific extensions (vulnerability modules, mutators, payloads), see Adding Extensions.

Prerequisites

System Requirements

WSHawk requires the following base system components:

| Component | Requirement | Notes | |-----------|-------------|-------| | Python | >=3.8 | Supports versions 3.8, 3.9, 3.10, 3.11, 3.12, 3.13 | | Git | Any recent version | For repository cloning and version control | | Operating System | Linux, macOS, Windows | Playwright supports all major platforms | | Disk Space | ~500MB | Includes Playwright browser binaries | | Internet Access | Required | For dependency downloads and OAST testing |

The Python version range is defined in setup.py:39, which specifies python_requires=">=3.8". The classifiers in setup.py:32-36 enumerate all officially supported Python versions.

Required Knowledge

Contributors should be familiar with:

Python async/await programming patterns
WebSocket protocol basics
Git workflow (fork, branch, pull request)
Command-line operations

Sources: setup.py:32-39, CONTRIBUTING.md:15-40

Development Environment Setup Workflow

flowchart TD
    Start([Developer])
    
    Fork["Fork Repository<br/>github.com/regaan/wshawk"]
    Clone["Clone Fork<br/>git clone https://github.com/USER/wshawk.git"]
    Venv["Create Virtual Environment<br/>python -m venv venv"]
    Activate["Activate Environment<br/>source venv/bin/activate (Linux/Mac)<br/>venv\\Scripts\\activate (Windows)"]
    InstallReq["Install Requirements<br/>pip install -r requirements.txt"]
    InstallDev["Install Development Package<br/>pip install -e ."]
    Playwright["Install Playwright Browsers<br/>playwright install chromium"]
    Test["Run Test Suite<br/>python tests/test_modules_quick.py"]
    Verify["Verify CLI Commands<br/>wshawk --help"]
    Branch["Create Feature Branch<br/>git checkout -b feature/name"]
    Ready([Development Ready])
    
    Start --> Fork
    Fork --> Clone
    Clone --> Venv
    Venv --> Activate
    Activate --> InstallReq
    InstallReq --> InstallDev
    InstallDev --> Playwright
    Playwright --> Test
    Test --> Verify
    Verify --> Branch
    Branch --> Ready
    
    style Fork fill:#f9f9f9
    style Test fill:#f9f9f9
    style Ready fill:#e8e8e8

Development Environment Setup Workflow Diagram: This flowchart illustrates the complete sequence from initial repository fork through development readiness. Each step references specific commands and files that will be detailed in subsequent sections.

Sources: CONTRIBUTING.md:18-40, requirements.txt:1-20

Repository Setup

Forking the Repository

Navigate to https://github.com/regaan/wshawk
Click the "Fork" button in the top-right corner
Select your GitHub account as the fork destination

Cloning Your Fork

git clone https://github.com/YOUR_USERNAME/wshawk.git
cd wshawk

Replace YOUR_USERNAME with your actual GitHub username. This creates a local copy of your fork.

Adding Upstream Remote

Configure the original repository as an upstream remote to sync future changes:

git remote add upstream https://github.com/regaan/wshawk.git
git remote -v

This should display four remotes: origin (your fork) and upstream (original repository), each with fetch and push URLs.

Sources: CONTRIBUTING.md:18-22

Virtual Environment Creation

Python virtual environments isolate project dependencies from system-wide packages, preventing version conflicts.

Creating the Environment

python -m venv venv

This creates a venv/ directory containing an isolated Python installation. The directory is excluded from version control via .gitignore:25-28.

Platform-Specific Activation

Linux/macOS:

source venv/bin/activate

Windows (Command Prompt):

venv\Scripts\activate.bat

Windows (PowerShell):

venv\Scripts\Activate.ps1

After activation, your shell prompt should be prefixed with (venv), indicating the active environment.

Deactivation

To deactivate the environment when finished:

deactivate

Sources: CONTRIBUTING.md:24-28, .gitignore:25-28

Dependency Installation

WSHawk has two dependency categories: runtime dependencies and development dependencies.

Dependency Categories

flowchart LR
    subgraph "Runtime Dependencies"
        WS["websockets>=12.0<br/>(WebSocket client)"]
        PW["playwright>=1.40.0<br/>(Browser automation)"]
        AH["aiohttp>=3.9.0<br/>(HTTP client for OAST)"]
        YML["PyYAML>=6.0<br/>(Config parsing)"]
        MQTT["asyncio-mqtt>=0.16.0<br/>(MQTT protocol)"]
    end
    
    subgraph "Development Dependencies"
        PT["pytest>=7.4.0<br/>(Test framework)"]
        PTA["pytest-asyncio>=0.21.0<br/>(Async test support)"]
        CLR["colorama>=0.4.6<br/>(Terminal colors)"]
    end
    
    subgraph "Code Modules"
        Scanner["scanner_v2.py<br/>WSHawkV2 class"]
        Verifier["xss_verifier.py<br/>Playwright usage"]
        OAST["oast_integration.py<br/>aiohttp usage"]
        Config["config loaders<br/>PyYAML usage"]
        Tests["tests/<br/>pytest usage"]
    end
    
    WS --> Scanner
    PW --> Verifier
    AH --> OAST
    YML --> Config
    PT --> Tests
    PTA --> Tests
    
    style WS fill:#f9f9f9
    style PW fill:#f9f9f9
    style PT fill:#f9f9f9

Dependency Architecture Diagram: Maps external dependencies to their primary consumers in the codebase. Runtime dependencies support core functionality, while development dependencies enable testing and quality assurance.

Installing All Dependencies

pip install -r requirements.txt

This installs all packages listed in requirements.txt:1-20. The file includes:

| Dependency | Version | Purpose | |------------|---------|---------| | websockets | >=12.0 | Core WebSocket protocol implementation | | playwright | >=1.40.0 | Headless browser for XSS verification | | aiohttp | >=3.9.0 | Async HTTP client for OAST callbacks | | PyYAML | >=6.0 | YAML configuration file parsing | | asyncio-mqtt | >=0.16.0 | MQTT protocol support | | pytest | >=7.4.0 | Test framework | | pytest-asyncio | >=0.21.0 | Async test support | | colorama | >=0.4.6 | Cross-platform terminal colors (optional) |

Note that setup.py:9-14 defines a minimal runtime subset (excludes dev dependencies like pytest).

Sources: requirements.txt:1-20, setup.py:9-14

Development Installation

Install WSHawk in editable mode to enable live code changes without reinstallation:

pip install -e .

The -e (editable) flag creates a symbolic link between the installation directory and your local source code. This allows modifications to .py files to take immediate effect.

What Gets Installed

This command processes setup.py:1-62 to:

Register CLI entry points (setup.py:41-48):
- wshawk → wshawk.__main__:cli
- wshawk-interactive → wshawk.interactive:cli
- wshawk-advanced → wshawk.advanced_cli:cli
- wshawk-defensive → wshawk.defensive_cli:cli
Include payload data files (setup.py:50-55):
- payloads/*.txt
- payloads/**/*.json
Install runtime dependencies (setup.py:40):
- Uses the minimal dependency list (excludes dev tools)

After installation, the CLI commands become globally available in the virtual environment.

Sources: CONTRIBUTING.md:31-34, setup.py:41-55

Playwright Browser Installation

Playwright requires browser binaries for headless testing. Install Chromium (used for XSS verification):

playwright install chromium

Browser Installation Details

| Browser | Size | Purpose | |---------|------|---------| | chromium | ~350MB | Primary browser for XSS verification | | firefox | ~400MB | Optional, not used by default | | webkit | ~300MB | Optional, not used by default |

Chromium is sufficient for WSHawk development. The binary installs to ~/.cache/ms-playwright/ (Linux/macOS) or %USERPROFILE%\AppData\Local\ms-playwright\ (Windows).

Verifying Installation

playwright --version

Expected output: Version 1.40.0 or higher.

The .playwright/ cache directory is excluded from version control via .gitignore:71.

Sources: requirements.txt:9, .gitignore:71

Running Tests

WSHawk includes a quick test suite for development validation.

Test Execution

python tests/test_modules_quick.py

This test file validates core module functionality. The test suite uses pytest with async support (requirements.txt:15-16).

Test Coverage Areas

flowchart TB
    TestSuite["tests/test_modules_quick.py"]
    
    subgraph "Core Modules"
        Scanner["scanner_v2.py<br/>WSHawkV2"]
        Intel["message_intelligence.py<br/>MessageIntelligence"]
        Fingerprint["server_fingerprinter.py<br/>ServerFingerprinter"]
        Payloads["payloads/<br/>Payload loading"]
    end
    
    subgraph "Vulnerability Modules"
        SQL["sql_injection.py<br/>SQLInjectionTester"]
        XSS["xss_detection.py<br/>XSSTester"]
        XXE["xxe_detection.py<br/>XXETester"]
        Session["session_hijacking.py<br/>SessionHijackingTester"]
    end
    
    subgraph "Defensive Modules"
        DNS["dns_exfiltration_test.py<br/>DNSExfiltrationTest"]
        Bot["bot_detection_test.py<br/>BotDetectionTest"]
        CSWSH["cswsh_test.py<br/>CSWSHTest"]
        WSS["wss_security_test.py<br/>WSSSecurityTest"]
    end
    
    TestSuite --> Scanner
    TestSuite --> Intel
    TestSuite --> Fingerprint
    TestSuite --> Payloads
    TestSuite --> SQL
    TestSuite --> XSS
    TestSuite --> XXE
    TestSuite --> Session
    TestSuite --> DNS
    TestSuite --> Bot
    TestSuite --> CSWSH
    TestSuite --> WSS
    
    style TestSuite fill:#f9f9f9

Test Coverage Diagram: Maps the test suite to code modules under validation. The quick test suite (test_modules_quick.py) provides fast feedback during development by testing core functionality without requiring live WebSocket endpoints.

Expected Test Output

Successful execution produces output similar to:

============================= test session starts ==============================
platform linux -- Python 3.11.0, pytest-7.4.0, pluggy-1.3.0
collected 15 items

tests/test_modules_quick.py ...............                              [100%]

============================== 15 passed in 2.45s ===============================

Test Artifacts

Tests generate temporary files excluded by .gitignore:50-82:

.pytest_cache/ - Pytest cache
test_results/ - Test output artifacts
wshawk_report_*.html - Generated scan reports

Sources: CONTRIBUTING.md:54-60, .gitignore:38-82

Installation Verification

Verify the complete installation by testing CLI commands and Python imports.

CLI Command Verification

Test all four entry points:

wshawk --help
wshawk-interactive --help
wshawk-advanced --help
wshawk-defensive --help

Each command should display its help text without errors. The entry points are registered via setup.py:42-47.

Python Import Verification

Test programmatic access:

python -c "from wshawk.scanner_v2 import WSHawkV2; print('Import successful')"

Expected output: Import successful

Module Discovery Verification

Verify package data inclusion:

python -c "from wshawk.__main__ import WSPayloads; p = WSPayloads(); print(f'Loaded {len(p.sqli_payloads)} SQL injection payloads')"

This confirms that payload files (setup.py:52-54) are correctly bundled and accessible.

Sources: setup.py:42-54, CONTRIBUTING.md:54-60

IDE Configuration (Optional)

Visual Studio Code

Create .vscode/settings.json (excluded via .gitignore:31):

{
    "python.defaultInterpreterPath": "${workspaceFolder}/venv/bin/python",
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": false,
    "python.linting.flake8Enabled": true,
    "python.formatting.provider": "black",
    "editor.formatOnSave": true,
    "files.exclude": {
        "**/__pycache__": true,
        "**/*.pyc": true,
        "**/*.egg-info": true
    }
}

PyCharm

Open project root directory
Navigate to: File → Settings → Project → Python Interpreter
Click gear icon → Add → Existing Environment
Select venv/bin/python (Linux/macOS) or venv\Scripts\python.exe (Windows)

Sources: .gitignore:30-36

Common Issues and Troubleshooting

Issue: `playwright` command not found

Cause: Playwright CLI not in PATH after installation.

Solution:

pip install playwright
playwright install chromium

Ensure virtual environment is activated (venv/bin/activate).

Issue: Import errors for `wshawk` modules

Cause: Package not installed in editable mode.

Solution:

pip install -e .

The -e flag is required for development to enable live code changes.

Issue: Missing payload files

Cause: Package data not included during installation.

Solution: Verify setup.py:49-55 specifies include_package_data=True and package_data dictionary. Reinstall:

pip uninstall wshawk
pip install -e .

Issue: Test failures with WebSocket connection errors

Cause: Tests attempting to connect to non-existent WebSocket servers.

Solution: The test_modules_quick.py suite should not require live connections. If tests fail, check test isolation and mocking.

Issue: Permission errors on Windows during Playwright installation

Cause: Insufficient permissions for browser binary installation.

Solution: Run command prompt as Administrator or modify Playwright cache location:

set PLAYWRIGHT_BROWSERS_PATH=%USERPROFILE%\playwright-browsers
playwright install chromium