diff --git a/.paul/codebase/architecture.md b/.paul/codebase/architecture.md new file mode 100644 index 0000000..d0229ec --- /dev/null +++ b/.paul/codebase/architecture.md @@ -0,0 +1,126 @@ +# Architecture — rank24.pl + +## Pattern +Custom PHP MVC — no framework. Layered as **controls → factory → view**, with Savant3 templates. + +## Entry Points + +| File | Purpose | +|------|---------| +| `index.php` | Front controller — bootstraps app, routes all page requests | +| `ajax.php` | xajax AJAX endpoint | +| `json.php` | JSON responses for AJAX calls | +| `api.php` | External API integration endpoint | +| `cron.php` | Background job runner | +| `proxy.php` | Proxy management endpoint | +| `ajax-check.php` | Health check / secondary AJAX | +| `dsf.php` | DataForSEO callback handler | + +## Directory Structure + +``` +rank24.pl/ +├── index.php Front controller & bootstrap +├── config.php All configuration (DB, proxy, intervals) +├── cron.php Cron job runner +├── .htaccess URL rewriting rules +│ +├── autoload/ PSR-0-style autoloaded classes +│ ├── class.*.php Core utility classes +│ ├── controls/ Page controllers (routing + business logic) +│ ├── factory/ Data access layer (DB queries via Medoo) +│ ├── view/ View renderers (build Savant3 template output) +│ ├── savant3/ Savant3 template engine internals +│ └── opd*/ Legacy PDO debug wrapper +│ +├── templates/ PHP view templates +│ ├── page/ Admin main/unlogged layouts +│ ├── client/ Client dashboard templates +│ ├── ranker/ Admin/ranker management templates +│ ├── reseller/ Reseller dashboard templates +│ ├── html/ Reusable form component templates +│ ├── other/ Pagination, alerts, misc templates +│ └── cron/ Cron management view +│ +├── functions/ xajax AJAX handler functions +│ ├── xajax.php xajax init & config +│ ├── xajax-ranker.php Rank-related AJAX +│ ├── xajax-messages.php Messaging AJAX +│ ├── xajax-analysis.php Analysis AJAX +│ └── xajax-settings.php Settings AJAX +│ +├── libraries/ Third-party libraries +│ ├── medoo.php ORM +│ ├── grid/ Custom DataTables grid +│ └── framework/ Bootstrap + jQuery + 57 plugins +│ +├── layout/ Static assets, compiled CSS +├── resources/ xajax, phpmailer, mPDF +└── temp/, temp_t/ File cache directories +``` + +## Autoloading + +Defined in `index.php`: +```php +// namespace\ClassName → autoload/namespace/class.ClassName.php +// ClassName → autoload/class.ClassName.php +spl_autoload_register('__autoload_my_classes'); +``` + +## Request Flow + +``` +HTTP request + → .htaccess: rewrites /m/a/params → ?module=m&action=a¶ms + → index.php: load config, init $db (OPD) + $mdb (Medoo), start session, init $cache + → \controls\Page::checkUrlParams() (handles ?rw= special actions) + → \controls\Page::getContent() (resolves module+action → controller class) + → \controls\[Module]::method() (business logic, calls factory) + → \factory\[Module]::query() (Medoo DB queries) + → \view\[Module]::render() (assigns data to Savant3, returns HTML) + → \view\Page::show() (wraps in role-appropriate layout) + → HTML → browser +``` + +## User Roles & Routing + +| Role | Source table | Layout | +|------|-------------|--------| +| `admin` | `pro_users` | `templates/page/main-layout.php` | +| `client` | `pro_rr_clients` (type=0) | `templates/client/main-layout.php` | +| `worker` | `pro_rr_clients` (type=2) | `templates/client/main-layout.php` | +| `reseller` | `pro_rr_clients` (type=1) | `templates/reseller/main-layout.php` | + +## Template System + +Two systems co-exist: + +**Savant3** (primary): `$tpl = new \Savant3; $tpl->varName = $val; return $tpl->fetch('ranker/summary');` + +**Tpl** (lightweight): `$tpl = new \Tpl; $tpl->render('other/pager');` +- Template search order: `templates_a/` → `templates_b/` → `templates/` + +## Database Layer + +Two connections initialized in `index.php`: +- `$db` — OPD (legacy PDO debug wrapper), used in older code paths +- `$mdb` — Medoo ORM, used in all newer code + +Factory pattern: controllers call `\factory\Ranker::getClient($id)` which does `global $mdb; return $mdb->get(...)`. + +## Cron System + +`cron.php` → `\Cron::staticMethod()` — jobs defined as static methods in `autoload/class.Cron.php`: +- `fill_missing_positions()` — interpolates missing rank records +- `archive_positions()` / `archive_empty()` — data archiving +- `check_proxy()` — validates proxy pool +- `get_phrases_positions_dfs3()` / `post_phrases_positions_dfs3()` — DataForSEO rank fetching + +## AJAX System + +`functions/xajax.php` registers handler functions → `ajax.php` processes incoming xajax requests → returns DOM update commands to jQuery on client. + +## Caching + +`\FileCache` stores serialized data in `temp/` and `temp_t/`. Invalidated via `\S::deleteCache()`. diff --git a/.paul/codebase/concerns.md b/.paul/codebase/concerns.md new file mode 100644 index 0000000..15dd9cf --- /dev/null +++ b/.paul/codebase/concerns.md @@ -0,0 +1,97 @@ +# Concerns & Technical Debt — rank24.pl + +## Security — CRITICAL + +### Hardcoded Credentials (must fix before any public exposure) + +| Secret | Location | Risk | +|--------|---------|------| +| MySQL password | `config.php` lines 2-5 | Full DB access if repo leaked | +| FTP password | `.vscode/ftp-kr.json`, `.vscode/sftp.json` | Full server access | +| DataForSEO API key | `autoload/class.Cron.php` ~lines 160, 262, 354 | API abuse / billing fraud | +| SMTP password | `autoload/class.S.php` ~lines 293-300 | Email spoofing | + +**Remediation**: move all secrets to environment variables or a `.env` file excluded from VCS. + +### SQL Injection + +- `autoload/class.Cron.php` ~line 200: raw string concatenation in DELETE query +- `autoload/class.GoogleRank.php` lines 74, 96, 100, 136, 158, 162: raw string concat in UPDATE queries +- `autoload/class.DataBase.php` lines 15, 47, 82: mixed OPD with string building + +**Remediation**: use Medoo's parameterized methods or PDO `bindValue()` for all dynamic values. + +### Other Security Issues (MEDIUM) + +- **Weak password hashing**: `md5($pass1)` in `autoload/class.DataBase.php` line 31 — use `password_hash()` +- **No CSRF protection**: state-changing AJAX operations in `ajax.php` lack CSRF tokens +- **Path traversal**: `autoload/class.DataBase.php` ~line 57 — user-supplied `image_folder` concatenated into file path without validation +- **Client-supplied MIME type**: file type validation in `class.DataBase.php` checks `$file['type']` (attacker-controlled) +- **Insecure deserialization**: `@unserialize()` used in `autoload/class.FileCache.php` line 43 and `autoload/opd.statement.php` +- **XSS**: `\S::get()` reads raw `$_POST`/`$_GET` without sanitization; values reach HTML output in multiple templates + +## Technical Debt + +### God Classes + +- `autoload/class.S.php` — 700+ lines; handles sessions, email, DNS, CSV, URL, string utils, DB helpers. Should be split. +- `autoload/class.GoogleRank.php` — 300+ lines; proxy selection logic repeated 4+ times with no extraction. +- `autoload/class.Cron.php` — 400+ lines; hardcoded credentials, multiple large functions. + +### Code Duplication + +- Proxy selection + backoff UPDATE query repeated verbatim ~4 times in `class.GoogleRank.php` +- Google block-detection strings (`"Our systems have detected unusual traffic"`) duplicated in multiple methods + +### Global State Anti-Pattern + +Every class does `global $db, $mdb, $user, $config, $cache;` — no DI, no service container. Makes refactoring and testing very difficult. + +### Two ORM Layers + +Both `$db` (OPD) and `$mdb` (Medoo) are initialized and used. Older code paths use OPD raw queries; newer paths use Medoo. Inconsistent access patterns throughout. + +### Deprecated PHP Patterns + +- Old-style constructor: `function DataEdit()` in `autoload/class.DataEdit.php` line 32 (should be `__construct()`) +- `global` variable injection instead of constructor parameters +- Short open tags ``, `=`, `=>` operators; spaces inside `( )` for function calls +- **Arrays**: mix of long `array()` in config, short `[]` in Medoo queries +- **Short PHP tags**: templates use `varName` +- Tpl templates receive variables as assigned properties, accessed in template scope +- HTML helper components in `templates/html/` — generated via `\Html::form_text()`, `\Html::select()`, etc. +- All UI strings use Polish language + +## Language + +- All user-facing text, comments, variable names, and commit messages are in **Polish** diff --git a/.paul/codebase/db_schema.md b/.paul/codebase/db_schema.md new file mode 100644 index 0000000..b4fa34e --- /dev/null +++ b/.paul/codebase/db_schema.md @@ -0,0 +1,98 @@ +# Database Schema — rank24.pl + +Database: `host700513_rank24` (MySQL 5.x, charset utf8) + +> This document reflects table names and columns inferred from source code analysis. +> For authoritative schema, run: `SHOW CREATE TABLE ;` + +## Known Tables + +### `pro_users` +Admin user accounts. + +| Column | Type | Notes | +|--------|------|-------| +| `id` | int | PK | +| `login` | varchar | Username | +| `password` | varchar | MD5 hash (legacy) | +| `type` | varchar | Role: `admin` | + +### `pro_rr_clients` +Client, reseller, and worker accounts. + +| Column | Type | Notes | +|--------|------|-------| +| `id` | int | PK | +| `login` | varchar | | +| `password` | varchar | MD5 hash | +| `type` | int | 0=client, 1=reseller, 2=worker | +| `active` | int | | +| `reseller_id` | int | FK to self (reseller parent) | + +### `pro_proxy_servers` +HTTP proxy pool for scraping. + +| Column | Type | Notes | +|--------|------|-------| +| `id` | int | PK | +| `proxy` | varchar | `ip:port` | +| `bg` | int | Ban/backoff counter | +| `bgd` | datetime | Backoff until datetime | +| `used` | datetime | Last used timestamp | +| `enabled` | int | 1=active, 0=disabled | + +### `pro_rr_sites` +Monitored websites per client. + +| Column | Type | Notes | +|--------|------|-------| +| `id` | int | PK | +| `client_id` | int | FK → `pro_rr_clients.id` | +| `url` | varchar | Domain / URL | +| `active` | int | | + +### `pro_rr_sites_majestic` +Majestic metrics cache per site. + +| Column | Type | Notes | +|--------|------|-------| +| `site_id` | int | FK → `pro_rr_sites.id` | +| `trust_flow` | int | | +| `citation_flow` | int | | +| `external_backlinks` | int | | +| `ref_domains` | int | | +| `updated_at` | datetime | | + +### `phrase_positions_statistic` +Live ranking data (rolling 2 years). + +| Column | Type | Notes | +|--------|------|-------| +| `id` | int | PK | +| `phrase_id` | int | FK → phrases table | +| `date` | date | | +| `position` | int | Google rank position | +| `url` | varchar | Result URL | + +### `phrase_positions_archive` +Archived ranking data (older than 2 years). + +Same structure as `phrase_positions_statistic`. + +## Table Name Patterns + +All application tables use the `pro_` prefix: +- `pro_users` — admin users +- `pro_rr_clients` — clients / resellers / workers +- `pro_rr_sites` — tracked sites +- `pro_rr_sites_majestic` — Majestic metrics +- `pro_proxy_servers` — proxy pool +- `phrase_positions_statistic` — live positions (no prefix — legacy naming) +- `phrase_positions_archive` — archived positions + +## Notes + +- Passwords stored as **MD5** — upgrade to `password_hash()` / `password_verify()` is a known debt item +- Positions older than 2 years are moved from `phrase_positions_statistic` → `phrase_positions_archive` by `\Cron::archive_positions()` +- Missing positions (gaps in daily records) are interpolated by `\Cron::fill_missing_positions()` +- Proxy backoff: `bgd = NOW() + (bg * 15 MINUTE)` where `bg` increments on each failure diff --git a/.paul/codebase/integrations.md b/.paul/codebase/integrations.md new file mode 100644 index 0000000..508e539 --- /dev/null +++ b/.paul/codebase/integrations.md @@ -0,0 +1,72 @@ +# External Integrations — rank24.pl + +## DataForSEO API (Primary — Active) + +- **Purpose**: Google SERP rank checking — replaces direct Google scraping +- **Client v2**: `autoload/RestClient.php` +- **Client v3**: `autoload/RestClient3.php` +- **Credentials**: hardcoded in `autoload/class.Cron.php` (lines ~160, ~262, ~354) — `pyziak84@gmail.com` +- **Usage**: `\Cron::post_phrases_positions_dfs3()` submits tasks; `\Cron::get_phrases_positions_dfs3()` retrieves results +- **Auth**: HTTP Basic Auth over HTTPS + +## Majestic (Active) + +- **Purpose**: Domain authority metrics — TF (Trust Flow), CF (Citation Flow), backlinks, RefDomains +- **Data stored in**: `pro_rr_sites_majestic` table +- **Refresh interval**: `$config['site']['majestic_interval']` = 7 days +- **Integration point**: `api.php` + relevant factory methods + +## SEMstorm (Active) + +- **Purpose**: Keyword traffic / visibility data +- **Refresh interval**: `$config['site']['semstorm_interval']` = 1 day +- **Integration point**: `api.php` + +## Proxy Providers (Disabled) + +| Provider | API endpoint (stored in config) | Config key | Status | +|---------|-------------------------------|-----------|--------| +| ProxyMarket | `http://www.proxymarket.pl/api/get/...` | `proxymarket-api` | disabled | +| Proxy.Adding.pl | `http://proxy.adding.pl/apiproxy/...` | `adding-api` | disabled | + +Active proxies are managed internally in the `pro_proxy_servers` DB table. + +## Internal Proxy Pool + +- **Table**: `pro_proxy_servers` (columns: `id`, `proxy`, `bg`, `bgd`, `used`, `enabled`) +- **Rotation logic**: `ORDER BY used ASC LIMIT 1` (least-recently-used) +- **Backoff**: failed proxy gets `bgd = NOW() + (bg * 15 minutes)` cooldown +- **Validation**: `\Cron::check_proxy()` pings each proxy, marks invalid ones disabled +- **Config**: `$config['proxy']['sv-check']` = 2, `$config['proxy']['ht-check']` = 4, `$config['proxy']['s-version']` = 1.218 + +## Direct Google Scraping (Legacy — still in codebase) + +- **Classes**: `autoload/class.GoogleScraper.php`, `autoload/class.GoogleRank.php`, `autoload/class.GoogleSite.php` +- `GoogleScraper` — cURL-based scraper with 24 rotating user agents, proxy support, cookie handling +- `GoogleRank` — orchestrates proxy selection → scrape → parse position → store result +- Block detection: checks for `"Our systems have detected unusual traffic"`, `"Forbidden"`, etc. +- Status: likely superseded by DataForSEO but classes remain + +## PHPMailer / SMTP Email + +- **Library**: `resources/phpmailer/` (v5.2.15) +- **Credentials**: SMTP hardcoded in `autoload/class.S.php` (lines ~293-300) — `biuro@project-pro.pl` +- **Usage**: transactional emails, notifications, organizer email: `poczta@project-dc.pl` + +## mPDF — PDF Reports + +- **Library**: `resources/mpdf60/` (v6.0) +- **Usage**: PDF report generation, rendered via `templates/*/reports-pdf.php` + +## xajax — AJAX Framework + +- **Library**: `resources/xajax/` +- **Handler files**: `functions/xajax-*.php` +- **Entry point**: `ajax.php` + +## FTP/SFTP Deployment + +- **Tool**: VS Code FTP-KR extension +- **Host**: `host700513.hostido.net.pl` +- **Remote path**: `/public_html/` +- **Config files**: `.vscode/ftp-kr.json`, `.vscode/sftp.json` (credentials stored in plaintext) diff --git a/.paul/codebase/stack.md b/.paul/codebase/stack.md new file mode 100644 index 0000000..8b6f44d --- /dev/null +++ b/.paul/codebase/stack.md @@ -0,0 +1,55 @@ +# Stack — rank24.pl + +## Core Language & Runtime +- **PHP** (5.6+, no version pinned) — backend logic, no framework +- **MySQL 5.x** — primary data store (host: `localhost`, db: `host700513_rank24`) +- **Apache** with `mod_rewrite` — URL rewriting via `.htaccess` +- **Timezone**: `Europe/Warsaw`, encoding: `UTF-8` + +## PHP Libraries + +| Library | Version | Purpose | Path | +|---------|---------|---------|------| +| Medoo | 1.2.1 | ORM / query builder (MySQL) | `libraries/medoo.php` | +| Savant3 | — | Template engine | `autoload/Savant3.php` / `autoload/savant3/` | +| OPD (opdClass) | — | Legacy PDO wrapper with debug console | `autoload/opd.class.php` | +| PHPMailer | 5.2.15 | SMTP email sending | `resources/phpmailer/` | +| mPDF | 6.0 | Server-side PDF generation | `resources/mpdf60/` | +| xajax | — | Server-side AJAX request handler | `resources/xajax/` | +| RestClient | 2.0 | DataForSEO API v2 client | `autoload/RestClient.php` | +| RestClient3 | 3.0 | DataForSEO API v3 client | `autoload/RestClient3.php` | +| Custom Grid | — | DataTables-based grid (view/edit/upload) | `libraries/grid/` | + +## Frontend Stack + +| Library | Version | Purpose | +|---------|---------|---------| +| jQuery | 1.11.1 | DOM / AJAX | `libraries/framework/vendor/jquery/` | +| Bootstrap | 3.x | CSS framework, JS components | `libraries/framework/` | +| DataTables | — | Sortable/filterable tables | `libraries/framework/vendor/plugins/` | +| CKEditor | — | Rich text editing | `libraries/framework/vendor/plugins/` | +| Highcharts / C3 | — | Charts and data viz | `libraries/framework/vendor/plugins/` | +| Moment.js | — | Date manipulation | `libraries/framework/vendor/plugins/` | +| Select2 | — | Searchable selects | `libraries/framework/vendor/plugins/` | +| Dropzone | — | Drag-and-drop file uploads | `libraries/framework/vendor/plugins/` | +| Font Awesome / Glyphicons | — | Icon fonts | `libraries/framework/` | + +CSS preprocessing via **SCSS/SASS** — source in `layout/style-scss/`, compiled to `layout/style-css/custom.css`. + +## External Service Integrations + +| Service | Purpose | Config key | +|---------|---------|-----------| +| DataForSEO API v2/v3 | Google rank checking (primary) | credentials in `class.Cron.php` | +| Majestic | Domain metrics (TF, CF, backlinks) | `$config['site']['majestic_interval']` = 7d | +| SEMstorm | Keyword / traffic data | `$config['site']['semstorm_interval']` = 1d | +| ProxyMarket | Proxy provider (disabled) | `$config['proxy']['proxymarket-api']` | +| Proxy.Adding.pl | Proxy provider (disabled) | `$config['proxy']['adding-api']` | +| Internal proxy pool | HTTP proxies for scraping | table `pro_proxy_servers` | + +## Deployment + +- **FTP/SFTP** to `host700513.hostido.net.pl` — auto-upload via VS Code FTP-KR extension +- Remote root: `/public_html/` +- Config: `.vscode/ftp-kr.json`, `.vscode/sftp.json` +- No CI/CD pipeline; manual or FTP-based deploys only diff --git a/.paul/codebase/testing.md b/.paul/codebase/testing.md new file mode 100644 index 0000000..e15b13b --- /dev/null +++ b/.paul/codebase/testing.md @@ -0,0 +1,33 @@ +# Testing — rank24.pl + +## Test Infrastructure + +**No automated tests exist.** There are no: +- PHPUnit configuration files (`phpunit.xml`, `phpunit.xml.dist`) +- Test directories (`tests/`, `test/`, `spec/`) +- Test class files (`*Test.php`, `*Spec.php`) +- CI/CD pipeline that runs tests + +## How Testing Is Done + +All testing is **manual** via the web interface: + +1. **Web UI testing** — load pages in browser, exercise forms and interactions +2. **AJAX endpoints** — `ajax-check.php` used for health checks / spot validation +3. **Cron output** — `cron.php` returns JSON; check via browser or log inspection +4. **PHP error logs** — `error_reporting` is suppressed in production; errors must be caught via server logs or temporary enabling of reporting + +## Debugging Aids + +- **OPD debug console** — `$config['db']['debug']` enables query logging via `opd.debug.php` +- **`\S::pre()`** — utility for dumping variables (calls `var_dump` / `print_r`), used inline and commented out: `//\S::pre($results); exit;` +- **`file_put_contents('google-rank.txt', $result)`** — ad-hoc scraping debug log in `class.GoogleRank.php` +- **`$debbbb = $data`** — debug variable left in `class.S.php` + +## Adding Tests (Guidance for Future) + +If automated testing is introduced: +- PHPUnit is the natural choice for PHP +- Heavy global state (`$db`, `$mdb`, `$user`) makes unit testing hard without refactoring — start with integration tests against a test database +- Factory classes (`factory\*`) are the best seam for testing — they have clear input/output and centralize DB access +- Cron methods in `class.Cron.php` return `['status' => 'ok'|'empty', 'msg' => '...']` — easily assertable