This commit is contained in:
2026-05-06 23:19:35 +02:00
parent 34e6b6373f
commit b1b5e416ba
16 changed files with 1448 additions and 65 deletions

View File

@@ -0,0 +1,109 @@
# Architecture
## Entry points
| File | Purpose |
|---|---|
| [index.php](index.php) | Main router. Autoloader (lines 3-12), session init, `$route_aliases` table (lines 40-59), URL-segment fallback (lines 70-79), auth via session or persistent cookie (lines 88-102), public-path whitelist for `/api/*`, `/cron/*`, `/login` (lines 106-118). Default module: `campaigns/main_view`. |
| [ajax.php](ajax.php) | AJAX handler. Same autoloader. Session regeneration on first request, IP binding check (lines 28-33). Responses: `echo json_encode(...); exit`. |
| [api.php](api.php) | External API endpoints (~1350 lines). Uses RedBeanPHP. Helpers `api_json_response()` and `api_validate_api_key()`. |
| [cron.php](cron.php) | Legacy cron. Calls `\Cron::tasks_emails()` and `\Cron::recursive_tasks()`. |
Modern cron routes (dispatched through index.php → `\controls\Cron`):
| Route | Purpose |
|---|---|
| `/cron/cron_universal` | Google Ads campaigns + products daily snapshot |
| `/cron/cron_campaigns_product_alerts_merchant` | Product alerts from Merchant Center |
| `/cron/cron_products_urls` | Bulk fetch product URLs |
| `/cron/cron_facebook_ads` | Facebook Ads sync (30-day window) |
| `/cron/cron_xml_feed_import` | Import supplemental/product XML feeds |
## Routing
`index.php` builds a request URL into `$_GET['module']` and `$_GET['action']`:
1. Apply `$route_aliases` map (e.g. `/login``users/login_form`).
2. Fallback: `/$seg0/$seg1``module=$seg0`, `action=$seg1`.
3. Default: `campaigns/main_view`.
4. `\controls\Site::route()` instantiates `\controls\{Module}` and calls action method.
## Layers
### Controllers — `autoload/controls/` (namespace `\controls`)
Static action methods. Pattern:
```php
$id = (int) \S::get('client_id');
$rows = \factory\Campaigns::get_campaigns_list($id);
echo json_encode($rows); exit; // AJAX
return \Tpl::view('campaigns/main_view', ['clients' => $rows]); // page
```
Representative files: [autoload/controls/class.Campaigns.php](autoload/controls/class.Campaigns.php), [autoload/controls/class.Products.php](autoload/controls/class.Products.php), [autoload/controls/class.Cron.php](autoload/controls/class.Cron.php) (~5,200 lines — see [concerns.md](concerns.md)).
### Factories — `autoload/factory/` (namespace `\factory`)
Static methods wrapping `$mdb` queries. Examples:
- [autoload/factory/class.Campaigns.php](autoload/factory/class.Campaigns.php) (~400 lines)
- [autoload/factory/class.Products.php](autoload/factory/class.Products.php) (~1,540 lines)
- [autoload/factory/class.Logs.php](autoload/factory/class.Logs.php)
### Services — `autoload/services/` (namespace `\services`)
External API integrations (see [integrations.md](integrations.md)).
### Views — `autoload/view/` (namespace `\view`)
Thin orchestrators. [autoload/view/class.Site.php](autoload/view/class.Site.php) wraps controller output in `site/layout-logged.php`, injecting `campaign_alerts_count`, `user`, `current_module`, flash alerts.
### Templates — `templates/{module}/`
Rendered via `\Tpl::view('module/file', $data)`. Lookup order: `templates_user/` (override) → `templates/`. Data accessed in templates as `$this->varName` (magic `__get`). Output captured with `ob_start/ob_get_clean`.
## Modules
| Module | Controller | Purpose |
|---|---|---|
| campaigns | `\controls\Campaigns` | Google Ads campaign list, history, charts, alerts |
| products | `\controls\Products` | Merchant Center product feed, AI title/desc suggestions |
| clients | `\controls\Clients` | Merchant accounts (Google Ads ID, Merchant ID, settings) |
| users | `\controls\Users` | Auth, settings, API key, cron status dashboard |
| feeds | `\controls\Feeds` | Supplemental TSV feed generation |
| logs | `\controls\Logs` | System event log viewer |
| campaign_alerts | `\controls\CampaignAlerts` | AI-detected campaign issues |
| campaign_terms | `\controls\CampaignTerms` | Search term aggregation, keyword suggestions |
| facebook_ads | `\controls\FacebookAds` | Facebook Ads sync and tracking |
| allegro | `\controls\Allegro` | Allegro.pl marketplace integration (legacy) |
| cron | `\controls\Cron` | Cron execution dispatcher and status UI |
| site | layout only | `layout-logged.php`, `layout-unlogged.php` |
| html | components | Reusable form elements (input, textarea, select, etc.) |
## Database schema (selected)
Migrations in [migrations/](migrations/) (30+ files, e.g. `001_google_ads_settings.sql`). Tracked in `schema_migrations`. Run via `php install.php` (`--force`, `--with_demo`).
Key tables:
- `settings` — global key-value config store
- `clients` — merchant accounts (Google Ads Customer ID, Merchant ID)
- `campaigns`, `campaigns_history` — campaign metadata + daily snapshots
- `cron_sync_status` — pipeline phase tracking (pending → fetch → aggregate_30 → done)
- `campaign_alerts`, `campaign_search_terms_history`, `campaign_ad_groups`, `campaign_keywords`, `campaign_negative_keywords`
- `products`, `products_aggregate`, `products_keyword_planner_terms`, `products_merchant_sync_log`
- `logs` — structured event log (level/source/message/context JSON/client_id)
- `facebook_campaigns`, `facebook_campaigns_history`, `facebook_ad_sets`, `facebook_ads`, `facebook_ads_history`
## Request flow
**Page (HTML):** request → `.htaccess``index.php` → routing → auth → `\view\Site::show()``\controls\Site::route()` → controller action → factory → `\Tpl::view(...)` → wrapped in `site/layout-logged.php` → response.
**AJAX (JSON):** POST/GET → `ajax.php` → session+IP check → controller action → `echo json_encode(...); exit`.
**Cron (JSON):** external GET → `index.php` (whitelisted) → `\controls\Cron::cron_universal()` → service API call → write to `*_history` + `cron_sync_status``self::output_cron_response([...])`.
## Auth
Session-based, with persistent cookie. Cookie stores JSON `{email, hash}` (salted). On revisit, `index.php` lines 92-102 verify and rehydrate session. AJAX adds IP binding (`$_SESSION['ip']` vs `$_SERVER['REMOTE_ADDR']`); mismatch → `session_destroy()`.

146
.paul/codebase/concerns.md Normal file
View File

@@ -0,0 +1,146 @@
# Areas of Concern
Prioritized: **HIGH** = security/data risk, **MEDIUM** = significant tech debt, **LOW** = polish/quality.
---
## Security — HIGH
### Hardcoded credentials in [config.php](config.php)
DB password, email password, and remote DB host stored in plaintext. If the repo leaks (or is on a shared dev machine), full DB compromise is immediate.
**Fix:** move to `.env` + `getenv()`; add `config.php` to `.gitignore`; rotate the leaked secret.
### Unsafe `unserialize()`
PHP object-injection surface:
- [autoload/class.Cache.php](autoload/class.Cache.php) line 29 — `@unserialize($data)` (cache files in `temp/`)
- [libraries/grid/grid.php](libraries/grid/grid.php) lines 95, 122
- [libraries/medoo/medoo.php](libraries/medoo/medoo.php) line 1264
**Fix:** switch to `json_encode/decode`, or pass the `['allowed_classes' => false]` option.
### Insecure persistent-login cookie
[index.php](index.php) lines 92-102 sets a cookie containing JSON `{email, hash}` with no `HttpOnly`, no `Secure`, no `SameSite` (also see `setcookie()` calls in [autoload/controls/class.Users.php](autoload/controls/class.Users.php) lines 43, 561).
**Fix:** issue an opaque random token, store hash server-side, set `HttpOnly; Secure; SameSite=Strict`.
### `eval()` in vendored grid
[libraries/grid/templates/results.php](libraries/grid/templates/results.php) line 289 and [libraries/grid/templates/print.php](libraries/grid/templates/print.php) line 73 evaluate strings drawn from `$_SESSION`. Session takeover ⇒ RCE.
**Fix:** replace with a safe expression evaluator or simple template helpers.
### Debug scripts shipped with credentials
`tmp/debug_*.php` (7 files) embed live DB creds. If `tmp/` is web-accessible, they're a one-shot console.
**Fix:** delete from repo; add `tmp/` to `.gitignore`; verify `tmp/` is not served (check [.htaccess](.htaccess)).
---
## Security — MEDIUM
### No CSRF protection
POST endpoints (controllers, [ajax.php](ajax.php), [api.php](api.php)) accept requests with no token validation. **Fix:** generate per-session token, embed in forms, verify in mutating actions.
### Path traversal in `\Tpl::render()`
[autoload/class.Tpl.php](autoload/class.Tpl.php) lines 31-62 builds `include` paths from `$file` without whitelisting. If `$file` ever flows from request data, `../../config` is reachable. Currently template names are hardcoded in controllers, so risk is latent — keep it that way.
### Inconsistent XSS escaping in templates
[templates/products/main_view.php](templates/products/main_view.php) defines a local `escape_html()` but applies it inconsistently. Polish content with apostrophes / quoted JSON in `<script>` blocks needs `htmlspecialchars($v, ENT_QUOTES, 'UTF-8')` consistently.
### IP-bound sessions break legitimate users
[ajax.php](ajax.php) lines 28-33 destroys the session on any `REMOTE_ADDR` change. NAT, mobile networks, and VPN switches will log users out frequently. Consider fingerprinting (UA + IP `/24`) instead of strict IP equality.
### File upload in Allegro controller
[autoload/controls/class.Allegro.php](autoload/controls/class.Allegro.php) lines 47-62 validates extension via `pathinfo`, not MIME. **Fix:** `finfo_file()`; store outside webroot.
### `exec()` in vendored upload handler
[libraries/filemanager-9.14.1/UploadHandler.php](libraries/filemanager-9.14.1/UploadHandler.php) lines 1006, 1032. **Fix:** prefer the `imagick`/`gd` extension; if `exec` stays, validate every argument with `escapeshellarg`.
---
## Tech debt — MEDIUM
### Dual ORM (Medoo + RedBeanPHP)
Medoo (`$mdb`) is the documented standard, but [api.php](api.php) (~1350 lines) and [cron.php](cron.php) use RedBeanPHP (`\R::`). Two query styles, two error paths, two sets of edge cases. Estimated 2-3 weeks to consolidate to Medoo.
### Monolithic controllers / factories
| File | Lines |
|---|---|
| [autoload/controls/class.Cron.php](autoload/controls/class.Cron.php) | ~5,200 |
| [autoload/factory/class.Products.php](autoload/factory/class.Products.php) | ~1,540 |
| [api.php](api.php) | ~1,350 |
| [autoload/services/class.GoogleAdsApi.php](autoload/services/class.GoogleAdsApi.php) | ~3,200 |
`Cron` mixes campaign sync, product sync, search-term aggregation, and Facebook ingest. Extract per-pipeline service classes.
### N+1 queries in cron paths
[autoload/controls/class.Cron.php](autoload/controls/class.Cron.php) lines 96 (`task_user`), 155 (`users.email` inside nested loop). For 10 tasks × multiple users per task → ~100 round-trips. **Fix:** pre-fetch with `WHERE IN (...)`.
### Migration safety
[migrations/](migrations/) files use `IF NOT EXISTS` (good) and `schema_migrations` tracks application, but there's no rollback mechanism and no recovery path if a migration partially fails mid-run.
### Cache strategy gaps
[autoload/class.Cache.php](autoload/class.Cache.php) is file-based with MD5 keys, no invalidation hooks, no namespacing, no TTL audit. Suppression (`@unserialize`) hides corruption. Multi-server deployments would silently desync.
---
## Operational — MEDIUM
### No retry / backoff on external APIs
[autoload/services/class.GoogleAdsApi.php](autoload/services/class.GoogleAdsApi.php) issues 50+ direct `curl_exec` calls. No exponential backoff, no rate-limit detection. Same in [class.FacebookAdsApi.php](autoload/services/class.FacebookAdsApi.php). A throttle response just becomes a logged error.
### Global warning suppression
`error_reporting(E_ALL ^ E_NOTICE ^ E_STRICT ^ E_WARNING)` in [ajax.php](ajax.php) line 1 hides production bugs. Log warnings to a file, even if not displayed.
### No secrets rotation
API keys live in `settings` indefinitely; no expiry, no rotation reminder. Consider a 90-day reminder + log entry on access.
---
## Frontend — LOW
- jQuery 3.6 is fine; Bootstrap 5 (active) coexists with Bootstrap 4.1.3 (legacy) in [libraries/bootstrap-4.1.3/](libraries/bootstrap-4.1.3/) — verify nothing in production still loads it.
- Two Font Awesome versions (4.7.0 + 6.5.1) — same audit needed.
- SCSS is compiled manually. A 5-line `package.json` + `sass` watcher would prevent stale CSS commits.
---
## Compliance / privacy — LOW
- No documented GDPR data-export or deletion flow despite serving EU/PL users.
- Persistent-cookie design embeds user identifier; not strictly PII-violating but worth re-architecting alongside the cookie-security fix.
- No retention policy on `logs` or product/campaign history tables.
---
## Suggested remediation order
1. **Rotate and remove [config.php](config.php) credentials** — 1 day.
2. **Cookie hardening + opaque token** — 2-3 days.
3. **Replace `unserialize` with JSON in cache + grid** — 3-5 days.
4. **CSRF tokens on mutating endpoints** — 1 week.
5. **Consolidate ORMs (drop RedBeanPHP)** — 2-3 weeks.
6. **Split `Cron` and `Products` factory** — 2 weeks.
7. **Introduce PHPUnit + minimal CI** — 1 week.
Total realistic remediation: **~13 weeks** at one full-time engineer.

View File

@@ -0,0 +1,104 @@
# Coding Conventions
CLAUDE.md documents the conventions; actual code matches ~95%. Specific patterns:
## Naming
- **Files:** `class.{ClassName}.php` (consistent across 40+ classes).
- **Namespaces:** `\controls`, `\factory`, `\services`, `\view`. Root-namespace utilities live at `autoload/class.{Name}.php` (e.g. `\S`, `\Tpl`, `\DbModel`, `\Cache`, `\Chunk`, `\Cron`, `\Excel`, `\Html`).
- Mapping enforced by `spl_autoload_register` in [index.php](index.php) lines 3-12.
## Methods
- Heavy reliance on **static methods** in controllers, factories, services. ~99% of public API.
- Static modifier ordering inconsistent (`static public` vs `public static`).
## Database access
- Always via global `$mdb` (Medoo). Never `new` a connection.
- Initialized once in [index.php](index.php) lines 24-31.
- Raw SQL only for complex joins via `$mdb->query(':sql', [':param' => $value])->fetch()`.
- **Exception:** [api.php](api.php) and [cron.php](cron.php) use RedBeanPHP (`\R::`). See [concerns.md](concerns.md) for the inconsistency.
## Reading params
```php
$id = (int) \S::get('client_id');
```
`\S::get($name)` reads POST first, then GET. Defined in [autoload/class.S.php](autoload/class.S.php) lines 198-217.
**Violations found** (use `$_POST` / `$_GET` directly):
- [autoload/controls/class.Api.php](autoload/controls/class.Api.php) line 47 — API token fallback
- [autoload/controls/class.Products.php](autoload/controls/class.Products.php) — DataTables `$_POST['order'][0]['dir']`/`['name']`
## Response patterns
```php
// AJAX
echo json_encode([...]);
exit;
// Page
return \Tpl::view('module/template', ['var' => $value]);
```
100% consistent across controllers.
## Flash messages
```php
\S::alert('Nazwa klienta jest wymagana.');
header('Location: /clients');
exit;
```
`\S::alert()` writes to `$_SESSION['alert']`; layout reads it.
## Currency formatting
`\S::number_display($value)``"1 234,56 zł"` (Polish locale: space thousands separator, comma decimal). Defined at [autoload/class.S.php](autoload/class.S.php) lines 51-54.
## Localization
- All UI strings in **Polish**.
- Timezone fixed to `Europe/Warsaw` (`date_default_timezone_set` in [index.php](index.php) line 14, [ajax.php](ajax.php) line 13).
- Polish slug helper: `\S::seo($val)`. ASCII transliteration: `\S::noPL($val)`.
## Code style
- 2-space indent, no tabs.
- K&R braces (open on same line).
- No `declare(strict_types=1);` anywhere.
- No type hints on signatures.
- Explicit casting (`(int)`, `(string)`, `(float)`) used liberally for normalization.
## Comments
- Density low (~5%). No `@param`/`@return` docblocks in production code.
- Only [autoload/class.Chunk.php](autoload/class.Chunk.php) (vendored) has docblocks.
- Section headers occasionally in Polish (e.g. `// --- Autoryzacja ---`).
## Error handling
- Try/catch present in newer modules ([class.Clients.php](autoload/controls/class.Clients.php) lines 58-75, [class.Cron.php](autoload/controls/class.Cron.php)).
- Catches generic `\Throwable`; no custom exception classes.
- Errors persisted as DB settings (e.g. `google_ads_last_error`).
- No central logger; ad-hoc `error_log` and DB `logs` table.
- Global `error_reporting` suppresses `E_NOTICE | E_STRICT | E_WARNING` (set in [ajax.php](ajax.php) line 1).
## Templates
- Data injected via `\Tpl::view('module/file', ['k' => $v])`.
- Inside templates: `<?php echo $this->k ?>` (magic `__get`).
- Output buffering wraps each `include`.
- User overrides: `templates_user/` is searched before `templates/`.
## Adding a new module
1. Create `autoload/controls/class.{Module}.php` (namespace `\controls`).
2. Create `autoload/factory/class.{Module}.php` (namespace `\factory`).
3. Optionally create `autoload/view/class.{Module}.php` (namespace `\view`).
4. Create `templates/{module}/main_view.php` and other actions.
5. Add to `$route_aliases` in [index.php](index.php) if clean URL needed.
6. Add sidebar link in [templates/site/layout-logged.php](templates/site/layout-logged.php).

View File

@@ -0,0 +1,80 @@
# External Integrations
All API credentials are stored in the `settings` DB table and read via `\services\GoogleAdsApi::get_setting($key)` (used globally despite the name).
## Google Ads REST API
- **File:** [autoload/services/class.GoogleAdsApi.php](autoload/services/class.GoogleAdsApi.php) (~3,200 lines)
- **API version:** `v23` (constant `$API_VERSION = 'v23'`)
- **Base URLs:**
- Ads: `https://googleads.googleapis.com`
- Merchant Center: `https://shoppingcontent.googleapis.com/content/v2.1`
- OAuth: `https://oauth2.googleapis.com/token`
- **Auth:** OAuth 2.0 refresh-token flow. Settings keys: `google_ads_developer_token`, `google_ads_client_id`, `google_ads_client_secret`, `google_ads_refresh_token`, `google_ads_manager_account_id`. Merchant variant: `google_merchant_refresh_token` (falls back to ads token).
- **Endpoints used:**
- `POST /v23/customers/{customerId}/googleAds:search` — campaigns, search terms, ad groups
- `GET /content/v2.1/{merchantAccountId}/products` — product listing/details
- Product ID format: `online:{lang}:{feedLabel}:{offerId}`
- **Errors:** stored in settings keys `google_ads_last_error` / `google_ads_last_error_at`.
- **Cron:** `/cron/cron_universal`, `/cron/cron_products_urls`, `/cron/cron_campaigns_product_alerts_merchant`.
## Facebook Ads Graph API
- **File:** [autoload/services/class.FacebookAdsApi.php](autoload/services/class.FacebookAdsApi.php) (~300 lines)
- **API version:** `v25.0` (default, configurable per client)
- **Base URL:** `https://graph.facebook.com/{version}/{accountId}/insights`
- **Auth:** access token in `facebook_ads_access_token` setting.
- **Methods:** `get_insights()`, `get_campaigns()`, `parse_date_range()`. 30-day rolling window.
- **Errors:** `facebook_ads_last_error` / `_at`.
- **Cron:** `/cron/cron_facebook_ads`.
## Google Merchant Center
Integrated through `GoogleAdsApi`. Supplemental TSV feeds generated via [autoload/services/class.SupplementalFeed.php](autoload/services/class.SupplementalFeed.php) into `feeds/supplemental_{client_id}.tsv`. XML feed parsing via [autoload/services/class.XmlFeedImporter.php](autoload/services/class.XmlFeedImporter.php) using `\Chunk` streaming reader.
## OpenAI
- **File:** [autoload/services/class.OpenAiApi.php](autoload/services/class.OpenAiApi.php) (~400 lines)
- **Endpoint:** `https://api.openai.com/v1/chat/completions`
- **Auth:** Bearer token in `openai_api_key` setting; model in `openai_model`.
- **Use cases:** product title (≤150 chars) and description (≤5000 chars) generation, page-content fetch + HTML strip + LLM rewrite. System prompt: Polish-language Merchant Center best practices.
## Claude (Anthropic)
- **File:** [autoload/services/class.ClaudeApi.php](autoload/services/class.ClaudeApi.php) (~300 lines)
- **Endpoint:** `https://api.anthropic.com/v1/messages`
- **Default model:** `claude-sonnet-4-5-20250929` (override via `claude_model`)
- **Auth:** `x-api-key` header from `claude_api_key`. `anthropic-version: 2023-06-01`.
- **Use cases:** same as OpenAI (product text optimization).
## Google Gemini
- **File:** [autoload/services/class.GeminiApi.php](autoload/services/class.GeminiApi.php) (~400 lines)
- **Base:** `https://generativelanguage.googleapis.com/v1beta/models/`
- **Default model:** `gemini-2.5-flash` (override via `gemini_model`)
- **Auth:** API key in `?key=` query param (`gemini_api_key`).
- **Special handling:** detects `gemini-2.5*` as thinking models — multiplies max_tokens by 6 for internal reasoning. Uses `systemInstruction` field.
## SMTP (PHPMailer)
- **Library:** [libraries/phpmailer/](libraries/phpmailer/) (3 files)
- **Wrapper:** `\S::send_email($email, $subject, $text, $file)` in [autoload/class.S.php](autoload/class.S.php)
- **Config (from [config.php](config.php)):** host `mail.project-pro.pl`, port 25, login `www@project-pro.pl`, password in plaintext (see [concerns.md](concerns.md)). SSL/TLS verification disabled (self-signed cert support). UTF-8, HTML body. Default From `www@projectpro.pl`, Reply-To `biuro@project-pro.pl`.
## Other / minor
- **Allegro.pl** — [autoload/controls/class.Allegro.php](autoload/controls/class.Allegro.php) (legacy marketplace integration)
- **Open Page Rank API** — referenced in [api.php](api.php) for domain authority lookups
- **Domain tester** — third-party domain validation in [api.php](api.php)
## Summary
| Integration | API ver | Auth | Cron |
|---|---|---|---|
| Google Ads | v23 | OAuth 2.0 refresh | `/cron/cron_universal` |
| Google Merchant | v2.1 | OAuth 2.0 | `/cron/cron_campaigns_product_alerts_merchant` |
| Facebook Ads | v25.0 | Bearer token | `/cron/cron_facebook_ads` |
| OpenAI | latest | API key (header) | on-demand |
| Claude | v1 / 2023-06-01 | `x-api-key` | on-demand |
| Gemini | v1beta | API key (query) | on-demand |
| SMTP | — | Basic auth | reminders via `cron.php` |

View File

@@ -0,0 +1,53 @@
# adsPRO — Codebase Overview
**Generated:** 2026-05-06
## What it is
adsPRO is a PHP web application for managing **Google Ads**, **Facebook Ads**, and **Google Merchant Center** campaigns. It tracks performance (ROAS, budgets, conversions), manages product feeds, and provides AI-powered suggestions (OpenAI / Claude / Gemini) for product titles and descriptions. UI in Polish, timezone `Europe/Warsaw`.
## At a glance
| Aspect | Value |
|---|---|
| Language | PHP (no framework, custom MVC-like) |
| Database | MySQL via Medoo ORM (`$mdb`) + RedBeanPHP (`\R::`) in legacy paths |
| Frontend | jQuery 3.6, Bootstrap 5, DataTables 2.1.7, Highcharts, Select2, FontAwesome 6.5.1 |
| Build | SCSS → CSS (manual); no Composer, no npm |
| Tests | None (manual `tmp/debug_*.php` scripts) |
| CI/CD | None |
## Entry points
- [index.php](index.php) — main router, autoloader, auth, request dispatch
- [ajax.php](ajax.php) — AJAX endpoints, session+IP binding
- [api.php](api.php) — external API endpoints (~1350 lines, RedBeanPHP)
- [cron.php](cron.php) — legacy cron (task reminders, recurring tasks)
- `/cron/*` routes (via index.php) — modern cron pipelines
## Layered structure
- **`autoload/controls/`** — Controllers (static methods, read params via `\S::get()`)
- **`autoload/factory/`** — Data access (static, uses global `$mdb`)
- **`autoload/services/`** — External API integrations
- **`autoload/view/`** — Thin view helpers wrapping `\Tpl::view()`
- **`templates/{module}/`** — PHP templates accessed via `$this->varName`
- **`migrations/`** — Numbered SQL files, tracked in `schema_migrations` table
## Key base classes
| Class | File | Purpose |
|---|---|---|
| `\S` | [autoload/class.S.php](autoload/class.S.php) | Params (`get`), session, alerts, email, currency formatting |
| `\Tpl` | [autoload/class.Tpl.php](autoload/class.Tpl.php) | Template engine (looks in `templates_user/` then `templates/`) |
| `\DbModel` | [autoload/class.DbModel.php](autoload/class.DbModel.php) | Active Record base |
| `\Cache` | [autoload/class.Cache.php](autoload/class.Cache.php) | File-based cache in `temp/` |
## Documents in this map
- [tech-stack.md](tech-stack.md) — language, libraries, versions
- [architecture.md](architecture.md) — entry points, layers, data flow, modules
- [integrations.md](integrations.md) — external APIs (Google Ads, Facebook, AI providers, SMTP)
- [conventions.md](conventions.md) — coding style and patterns
- [testing.md](testing.md) — testing posture (limited)
- [concerns.md](concerns.md) — risks, security, tech debt

View File

@@ -0,0 +1,47 @@
# Tech Stack
## Backend
| Component | Version | Notes |
|---|---|---|
| PHP | 5.6+ assumed (no explicit constraint) | No `composer.json`, no `declare(strict_types)` |
| MySQL | unspecified | Schema in [migrations/](migrations/) |
| Medoo ORM | 1.7.10 | [libraries/medoo/medoo.php](libraries/medoo/medoo.php), used via global `$mdb` |
| RedBeanPHP | bundled | [libraries/rb.php](libraries/rb.php), used in [api.php](api.php) and [cron.php](cron.php) |
| PHPMailer | bundled | [libraries/phpmailer/](libraries/phpmailer/) |
**Autoloader:** custom `spl_autoload_register` in [index.php](index.php) maps namespaces to `autoload/{layer}/class.{Name}.php`. No Composer.
## Frontend
| Library | Version | Source |
|---|---|---|
| jQuery | 3.6.0 | CDN |
| jQuery UI | bundled | [libraries/framework/vendor/jquery/jquery_ui/](libraries/framework/vendor/jquery/jquery_ui/) |
| Bootstrap | 5 (active) + 4.1.3 (legacy) | CDN + [libraries/bootstrap-4.1.3/](libraries/bootstrap-4.1.3/) |
| DataTables | 2.1.7 | CDN |
| Highcharts | latest | CDN |
| Select2 | 4.1.0-rc.0 | CDN + [libraries/select2/](libraries/select2/) |
| Font Awesome | 6.5.1 (active) + 4.7.0 (legacy) | CDN + [libraries/font-awesome-4.7.0/](libraries/font-awesome-4.7.0/) |
| Moment.js | 2.18.1 | CDN + [libraries/moment/](libraries/moment/) |
| Date Range Picker | bundled | [libraries/daterange/](libraries/daterange/) |
| CKEditor | bundled | [libraries/ckeditor/](libraries/ckeditor/) |
Custom assets: [libraries/adspro-dialog.js](libraries/adspro-dialog.js), [libraries/functions.js](libraries/functions.js), [libraries/xlsxwriter.class.php](libraries/xlsxwriter.class.php).
## Styling
- SCSS: [layout/style.scss](layout/style.scss) → [layout/style.css](layout/style.css) (manual compilation; no automated build pipeline)
- Source map present: [layout/style.css.map](layout/style.css.map)
## Configuration
- [config.php](config.php) — DB credentials, SMTP settings (plaintext; see [concerns.md](concerns.md))
- `settings` DB table — runtime config, API keys, cron state, feature flags. Accessed via `\services\GoogleAdsApi::get_setting($key)` / `set_setting($k, $v)` (used globally despite the class name).
## Tooling absent
- No Composer / package.json / lockfiles
- No PHPUnit / phpcs / phpstan
- No `.editorconfig`, no `.github/workflows`, no CI
- No linting / formatting config

35
.paul/codebase/testing.md Normal file
View File

@@ -0,0 +1,35 @@
# Testing
## Posture
**No formal test infrastructure.** No PHPUnit, no Pest, no `tests/` or `spec/` directories, no `phpunit.xml`, no `composer.json`. No CI pipeline (no `.github/workflows`).
## What exists
Manual debug scripts under [tmp/](tmp/) — developer-run, not automated:
| File | Purpose |
|---|---|
| `tmp/debug_clients.php` | Client sync verification |
| `tmp/debug_clients_remote.php`, `debug_clients_remote2.php` | Remote API checks |
| `tmp/debug_dup_products.php` | Duplicate product detection (currently uncommitted) |
| `tmp/debug_eligible_remote.php` | Eligibility checks |
| `tmp/debug_products_urls.php` | URL fetch verification |
| `tmp/campaign_alerts_debug.log` | Output log from alerts run |
| `tmp/meta_active_last30d.json`, `tmp/products_data_save.txt` | Test data snapshots |
These scripts contain **live database credentials** — see [concerns.md](concerns.md) for the security implication.
## Implicit "tests"
- `cron_sync_status` table acts as production-time integration check (phase progression).
- `logs` table provides audit trail for after-the-fact debugging.
- Settings keys `*_last_error` / `*_last_error_at` capture last failure per integration.
## Recommendations (if testing is introduced)
1. Adopt PHPUnit via Composer (also fixes the lack of dependency management).
2. Start with **factory** layer — it's the most testable (pure DB queries given a fixture DB).
3. Service layer needs HTTP client abstraction before the cURL calls can be tested without hitting live APIs.
4. Migrations are already partly idempotent — a CI job that runs `php install.php` against a scratch DB would be cheap insurance.
5. Move `tmp/debug_*.php` content into proper integration tests; delete the scripts.

View File

@@ -0,0 +1,42 @@
{"ts":"2026-05-06T20:10:08Z","tool":"Bash","cmd":"mysql --version 2>&1 | head -3\",\"description\":\"Check mysql client availability\"},\"tool_response\":{\"stdout\":\"/usr/bin/bash: line 1: mysql: command not found\",\"stderr\":\"\",\"interrupted\":","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:10:14Z","tool":"Bash","cmd":"where.exe mysql 2>&1; where.exe php 2>&1\",\"description\":\"Check mysql/php on Windows\"},\"tool_response\":{\"stdout\":\"INFO: Could not find files for the given pattern(s).\\r\\nC:\\\\xampp\\\\php\","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:10:38Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\debug_dup_products.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:10:48Z","tool":"Edit","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\debug_dup_products.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:10:57Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/debug_dup_products.php\" 2>&1\",\"timeout\":60000,\"description\":\"Run diagnostic against remote DB\"},\"tool_response\":{\"stdout","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:11:58Z","tool":"Bash","cmd":"mkdir -p .paul/codebase && ls .paul/codebase/\",\"description\":\"Create codebase directory\"},\"tool_response\":{\"stdout\":\"\",\"stderr\":\"\",\"interrupted\":false,\"isImage\":false,\"noOutputExpe","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:12:53Z","tool":"Bash","cmd":"Get-ChildItem -Path \"C:\\\\visual studio code\\\\projekty\\\\adsPRO\" -Recurse -Force | Select-Object FullName | wc -l\"},\"tool_response\":{\"stdout\":\"/usr/bin/bash: line 1: Get-ChildItem: command","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:13:39Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\debug_dup_audit.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:13:45Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/debug_dup_audit.php\" 2>&1\",\"timeout\":120000,\"description\":\"Audit duplicates and FK landscape\"},\"tool_response\":{\"stdout\","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:13:52Z","tool":"Bash","cmd":"wc -l \"C:/visual studio code/projekty/adsPRO/cron.php\" \"C:/visual studio code/projekty/adsPRO/api.php\"\"},\"tool_response\":{\"stdout\":\" 77 C:/visual studio code/projekty/adsPRO/cron.php\\n 13","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:14:31Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\debug_dup_indexes.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:14:56Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/debug_dup_indexes.php\" 2>&1\",\"timeout\":90000,\"description\":\"Audit indexes on tables with product_id\"},\"tool_response\":{\"","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:15:17Z","tool":"Write","file":"c:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\.paul\\\\codebase\\\\overview.md","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:15:35Z","tool":"Write","file":"c:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\.paul\\\\codebase\\\\tech-stack.md","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:16:15Z","tool":"Write","file":"c:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\.paul\\\\codebase\\\\architecture.md","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:16:46Z","tool":"Write","file":"c:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\.paul\\\\codebase\\\\integrations.md","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:17:18Z","tool":"Write","file":"c:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\.paul\\\\codebase\\\\conventions.md","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:17:36Z","tool":"Write","file":"c:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\.paul\\\\codebase\\\\testing.md","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:18:26Z","tool":"Write","file":"c:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\.paul\\\\codebase\\\\concerns.md","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:18:42Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\merge_duplicate_products.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:26:43Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/merge_duplicate_products.php\" --dry-run 2>&1\",\"timeout\":600000,\"description\":\"Full dry-run\"},\"tool_response\":{\"stdout\":","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:28:44Z","tool":"Bash","cmd":"cd \"C:/visual studio code/projekty/adsPRO\" && BACKUP=\"backup_dup_$(date +%Y%m%d_%H%M).sql\" && \"C:/xampp/mysql/bin/mysqldump.exe\" -h host700513.hostido.net.pl -u host700513_adspro -p\"2Ug7DvBy5MC","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:28:55Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/merge_duplicate_products.php\" --execute --client=2 --limit=5 2>&1\",\"timeout\":120000,\"description\":\"Test execute on 5 groups\","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:29:11Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\debug_verify_test.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:29:17Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/debug_verify_test.php\" 2>&1\",\"timeout\":60000,\"description\":\"Verify test execute\"},\"tool_response\":{\"stdout\":\"=== Czy l","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:39:25Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/merge_duplicate_products.php\" --execute 2>&1\",\"timeout\":600000,\"description\":\"Full migration\"},\"tool_response\":{\"stdout\","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:40:09Z","tool":"Bash","cmd":"until grep -q \"STATYSTYKI (WYKONANO)\" \"C:/Users/jacek/AppData/Local/Temp/claude/C--visual-studio-code-projekty-adsPRO/002c2267-55dd-47de-b760-4f48b17e7e6c/tasks/bcrvifsv6.output\" 2>/dev/null; do s","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:45:18Z","tool":"Bash","cmd":"until grep -q \"STATYSTYKI (WYKONANO)\" \"C:/Users/jacek/AppData/Local/Temp/claude/C--visual-studio-code-projekty-adsPRO/002c2267-55dd-47de-b760-4f48b17e7e6c/tasks/bcrvifsv6.output\" 2>/dev/null; do s","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:45:39Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\debug_verify_final.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:45:46Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/debug_verify_final.php\" 2>&1\",\"timeout\":90000,\"description\":\"Final verification\"},\"tool_response\":{\"stdout\":\"=== Czy z","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:46:23Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\migrations\\\\031_products_unique_client_offer.sql","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:46:44Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\apply_migration_031.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:47:17Z","tool":"Write","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\tmp\\\\apply_migration_031.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:47:23Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" \"C:/visual studio code/projekty/adsPRO/tmp/apply_migration_031.php\" 2>&1\",\"timeout\":120000,\"description\":\"Apply migration 031\"},\"tool_response\":{\"stdout\":\"Apliku","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:49:48Z","tool":"Edit","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\autoload\\\\factory\\\\class.Products.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:49:57Z","tool":"Edit","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\autoload\\\\controls\\\\class.Cron.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:50:04Z","tool":"Edit","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\autoload\\\\controls\\\\class.Api.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:50:10Z","tool":"Edit","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\autoload\\\\controls\\\\class.Api.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:50:24Z","tool":"Edit","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\autoload\\\\controls\\\\class.Allegro.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:50:44Z","tool":"Edit","file":"C:\\\\visual studio code\\\\projekty\\\\adsPRO\\\\autoload\\\\services\\\\class.XmlFeedImporter.php","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:50:54Z","tool":"Bash","cmd":"\"C:/xampp/php/php.exe\" -l \"C:/visual studio code/projekty/adsPRO/autoload/factory/class.Products.php\" 2>&1; \"C:/xampp/php/php.exe\" -l \"C:/visual studio code/projekty/adsPRO/autoload/controls/cl","cwd":"/c/visual studio code/projekty/adsPRO"}
{"ts":"2026-05-06T20:51:06Z","tool":"Bash","cmd":"rm \"C:/visual studio code/projekty/adsPRO/tmp/debug_dup_products.php\" \"C:/visual studio code/projekty/adsPRO/tmp/debug_dup_audit.php\" \"C:/visual studio code/projekty/adsPRO/tmp/debug_dup_indexes.","cwd":"/c/visual studio code/projekty/adsPRO"}

View File

@@ -134,30 +134,22 @@ class Allegro {
$client_id = $mdb -> id();
}
if ( !$mdb -> count( 'products', [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer['offer_id'] ] ] ) )
{
$product_data = [
'client_id' => $client_id,
'offer_id' => $offer['offer_id'],
'name' => $offer['offer_name'],
];
$existing_id = (int) $mdb -> get( 'products', 'id', [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer['offer_id'] ] ] );
if ( $mdb -> insert( 'products', $product_data ) )
if ( !$existing_id )
{
$product_id = $mdb -> id();
$offers_added++;
}
$product_id = \factory\Products::ensure_product( $client_id, $offer['offer_id'], [ 'title' => $offer['offer_name'] ] );
if ( $product_id ) $offers_added++;
}
else
{
$product = $mdb -> get( 'products', [ 'id', 'name' ], [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer['offer_id'] ] ] );
$product = $mdb -> get( 'products', [ 'id', 'title' ], [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer['offer_id'] ] ] );
$product_id = $product['id'];
$offer_current_name = $product['name'];
$offer_current_name = $product['title'];
if ( $offer_current_name != $offer['offer_name'] and $offer['date_add'] == date( 'Y-m-d', strtotime( '-1 days', time() ) ) )
{
$mdb -> update( 'products', [ 'name' => $offer['offer_name'] ], [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer['offer_id'] ] ] );
$mdb -> update( 'products', [ 'title' => $offer['offer_name'] ], [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer['offer_id'] ] ] );
$mdb -> insert( 'products_comments', [ 'product_id' => $product_id, 'comment' => 'Zmiana nazwy oferty na: ' . $offer['offer_name'], 'type' => 2, 'date_add' => $offer['date_add'] ] );
}
}

View File

@@ -398,24 +398,7 @@ class Api
{
$offer_data = [];
if ( !$mdb -> count( 'products', [ 'AND' => [ 'client_id' => $data['client_id'], 'offer_id' => $offer['OfferId'] ] ] ) )
{
$offer_data['client_id'] = $data['client_id'];
$offer_data['offer_id'] = $offer['OfferId'];
$offer_data['offer_name'] = $offer['ProductTitle'];
$mdb -> insert( 'products', [
'client_id' => $data['client_id'],
'offer_id' => $offer['OfferId'],
'name' => $offer['ProductTitle']
] );
$offer_id = $mdb -> id();
}
else
{
$offer_id = $mdb -> get( 'products', 'id', [ 'AND' => [ 'client_id' => $data['client_id'], 'offer_id' => $offer['OfferId'] ] ] );
}
$offer_id = \factory\Products::ensure_product( $data['client_id'], $offer['OfferId'], [ 'title' => $offer['ProductTitle'] ] );
if ( $offer_id )
{
@@ -520,19 +503,7 @@ class Api
$product_title = $offer_external_id;
}
if ( !$mdb -> count( 'products', [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer_external_id ] ] ) )
{
$mdb -> insert( 'products', [
'client_id' => $client_id,
'offer_id' => $offer_external_id,
'name' => $product_title
] );
$product_id = $mdb -> id();
}
else
{
$product_id = $mdb -> get( 'products', 'id', [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer_external_id ] ] );
}
$product_id = \factory\Products::ensure_product( $client_id, $offer_external_id, [ 'title' => $product_title ] );
if ( !$product_id )
{

View File

@@ -1444,13 +1444,7 @@ class Cron
if ( !$existing_product )
{
$mdb -> insert( 'products', [
'client_id' => $client_id,
'offer_id' => $offer_external_id,
'title' => $product_title
] );
$product_id = $mdb -> id();
$product_id = \factory\Products::ensure_product( $client_id, $offer_external_id, [ 'title' => $product_title ] );
$products_by_offer_id[ $offer_external_id ] = [
'id' => (int) $product_id,

View File

@@ -21,6 +21,38 @@ class Products
return true;
}
/**
* Bezpieczne wstawienie/odczyt produktu po (client_id, offer_id).
* Wymaga UNIQUE KEY uk_products_client_offer (zob. migracja 031).
* Zwraca product_id istniejacego lub nowo wstawionego wiersza.
* Odporne na race condition: przy bledzie 1062 (Duplicate entry) robi SELECT.
*/
static public function ensure_product( $client_id, $offer_id, array $insert_data = [] )
{
global $mdb;
$client_id = (int) $client_id;
$offer_id = trim( (string) $offer_id );
if ( $client_id <= 0 || $offer_id === '' ) return 0;
$existing_id = (int) $mdb -> get( 'products', 'id', [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer_id ] ] );
if ( $existing_id > 0 ) return $existing_id;
$row = array_merge( $insert_data, [ 'client_id' => $client_id, 'offer_id' => $offer_id ] );
try
{
$mdb -> insert( 'products', $row );
$new_id = (int) $mdb -> id();
if ( $new_id > 0 ) return $new_id;
}
catch ( \PDOException $e )
{
if ( strpos( (string) $e -> getMessage(), '1062' ) === false ) throw $e;
}
return (int) $mdb -> get( 'products', 'id', [ 'AND' => [ 'client_id' => $client_id, 'offer_id' => $offer_id ] ] );
}
static public function delete_products( $product_ids ) {
global $mdb;
if ( empty( $product_ids ) || !is_array( $product_ids ) ) {

View File

@@ -439,6 +439,8 @@ class XmlFeedImporter
}
}
else
{
try
{
$insert_stmt -> execute( [
':client_id' => $client_id,
@@ -449,9 +451,25 @@ class XmlFeedImporter
':price' => $price,
] );
$inserted_count++;
if ( $is_debug_offer )
if ( $is_debug_offer ) $report['debug_offer']['inserted'] = true;
}
catch ( \PDOException $pe )
{
$report['debug_offer']['inserted'] = true;
// 1062 = race condition z UNIQUE KEY (client_id, offer_id) - aktualizuj zamiast wstawiac
if ( strpos( (string) $pe -> getMessage(), '1062' ) === false ) throw $pe;
$existing_id = (int) $pdo -> query( 'SELECT id FROM products WHERE client_id = ' . (int) $client_id . ' AND offer_id = ' . $pdo -> quote( (string) $item['offer_id'] ) . ' LIMIT 1' ) -> fetchColumn();
if ( $existing_id > 0 )
{
$update_stmt -> execute( [
':title' => $title,
':description' => $desc,
':custom_label_1' => $cl1,
':price' => $price,
':id' => $existing_id,
] );
$updated_count++;
if ( $is_debug_offer ) $report['debug_offer']['updated_via_race_fallback'] = true;
}
}
}
}

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,35 @@
-- Migracja: UNIQUE KEY na (client_id, offer_id) w tabeli `products`.
-- Cel: zapobiec ponownemu powstaniu duplikatow tego samego produktu (ten sam offer_id u tego samego klienta) wstawianych przez rozne sciezki importu (Cron sync z Google Ads, XmlFeedImporter, Api, Allegro).
-- Wymaga: wczesniejszego scalenia historycznych duplikatow (zob. tmp/merge_duplicate_products.php). Bez tego ALTER zwroci blad 1062.
-- Idempotentnosc: warunek przez INFORMATION_SCHEMA.
-- 1. Usun istniejacy NON-UNIQUE indeks `idx_products_client_offer` zeby zwolnic nazwe i nie trzymac dwoch indeksow na tej samej parze kolumn.
SET @sql = IF(
EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'products'
AND INDEX_NAME = 'idx_products_client_offer'
AND NON_UNIQUE = 1
),
'ALTER TABLE `products` DROP INDEX `idx_products_client_offer`',
'DO 1'
);
PREPARE stmt FROM @sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
-- 2. Dodaj UNIQUE KEY (client_id, offer_id). Pomijamy gdy juz istnieje indeks o tej nazwie.
SET @sql = IF(
EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'products'
AND INDEX_NAME = 'uk_products_client_offer'
),
'DO 1',
'ALTER TABLE `products` ADD UNIQUE KEY `uk_products_client_offer` (`client_id`, `offer_id`)'
);
PREPARE stmt FROM @sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;

View File

@@ -0,0 +1,290 @@
<?php
/**
* Migrator: scala duplikaty w tabeli `products` powstale przez wielokrotne importy.
*
* Reguly:
* - Grupowanie duplikatow: (client_id, TRIM(offer_id)).
* - Winner per grupa: ORDER BY clicks_all_time DESC, impressions_all_time DESC,
* agg_rows DESC, id ASC.
* - Strategia: WINNER KEEPS ALL.
* Tabele z UNIQUE zawierajacym product_id: usuwamy z losera te wiersze
* ktore konflikuja z winnerem, reszte przepinamy UPDATE.
* Tabele bez UNIQUE na product_id: zwykly UPDATE.
* - Wszystko per-loser w transakcji.
*
* Tryby uruchomienia:
* php merge_duplicate_products.php --dry-run (domyslny - tylko liczy)
* php merge_duplicate_products.php --execute (faktyczne zmiany)
* php merge_duplicate_products.php --execute --client=2 (ograniczenie do klienta)
*/
error_reporting(E_ALL);
ini_set('display_errors', '1');
chdir(__DIR__ . '/..');
require 'config.php';
require 'libraries/medoo/medoo.php';
$opts = getopt('', ['dry-run', 'execute', 'client::', 'limit::']);
$DRY_RUN = !isset($opts['execute']);
$CLIENT_ID = isset($opts['client']) ? (int) $opts['client'] : 0;
$LIMIT = isset($opts['limit']) ? (int) $opts['limit'] : 0;
$mdb = new medoo([
'database_type' => 'mysql',
'database_name' => $database['name'],
'server' => $database['remote_host'],
'username' => $database['user'],
'password' => $database['password'],
'charset' => 'utf8'
]);
$pdo = $mdb->pdo;
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
echo str_repeat('=', 70) . "\n";
echo " TRYB: " . ($DRY_RUN ? 'DRY-RUN (zadne zmiany nie beda zapisane)' : 'EXECUTE (zmiany zostana zapisane)') . "\n";
echo " KLIENT: " . ($CLIENT_ID > 0 ? $CLIENT_ID : 'WSZYSCY') . "\n";
echo " LIMIT GRUP: " . ($LIMIT > 0 ? $LIMIT : 'BRAK') . "\n";
echo str_repeat('=', 70) . "\n";
// === 1. Pobierz wszystkie grupy duplikatow ===
$where = "TRIM(COALESCE(offer_id,'')) <> ''";
if ($CLIENT_ID > 0) $where .= " AND client_id = " . $CLIENT_ID;
$groups_sql = "
SELECT client_id, TRIM(offer_id) AS oid, COUNT(*) AS cnt, GROUP_CONCAT(id ORDER BY id) AS ids
FROM products
WHERE $where
GROUP BY client_id, TRIM(offer_id)
HAVING cnt > 1
ORDER BY client_id, oid
";
if ($LIMIT > 0) $groups_sql .= " LIMIT " . $LIMIT;
$groups = $pdo->query($groups_sql)->fetchAll(PDO::FETCH_ASSOC);
echo "Grup duplikatow: " . count($groups) . "\n";
if (empty($groups)) {
echo "Brak duplikatow do scalenia.\n";
exit(0);
}
// === 2. Statystyki sumaryczne ===
$stat = [
'groups' => count($groups),
'losers_total' => 0,
'pa_conflicts_to_delete' => 0,
'pa_to_repoint' => 0,
'pt_conflicts_to_delete' => 0,
'pt_to_repoint' => 0,
'ph_conflicts_to_delete' => 0,
'ph_to_repoint' => 0,
'ph30_conflicts_to_delete' => 0,
'ph30_to_repoint' => 0,
'kpt_conflicts_to_delete' => 0,
'kpt_to_repoint' => 0,
'comments_to_repoint' => 0,
'alerts_to_repoint' => 0,
'sync_logs_to_repoint' => 0,
'products_to_delete' => 0,
];
// === 3. Iteracja ===
$processed = 0;
foreach ($groups as $g) {
$client_id = (int) $g['client_id'];
$oid = $g['oid'];
$ids = array_map('intval', explode(',', $g['ids']));
// wybor winnera
$rank_sql = "
SELECT p.id,
COALESCE(SUM(pa.clicks_all_time),0) AS tot_clicks,
COALESCE(SUM(pa.impressions_all_time),0) AS tot_impr,
COUNT(pa.id) AS agg_rows
FROM products p
LEFT JOIN products_aggregate pa ON pa.product_id = p.id
WHERE p.id IN (" . implode(',', $ids) . ")
GROUP BY p.id
ORDER BY tot_clicks DESC, tot_impr DESC, agg_rows DESC, p.id ASC
";
$ranked = $pdo->query($rank_sql)->fetchAll(PDO::FETCH_ASSOC);
$winner_id = (int) $ranked[0]['id'];
$loser_ids = array_filter($ids, fn($i) => $i !== $winner_id);
$stat['losers_total'] += count($loser_ids);
foreach ($loser_ids as $loser_id) {
try {
if (!$DRY_RUN) $pdo->beginTransaction();
// --- products_aggregate: UNIQUE (product_id, campaign_id, ad_group_id) ---
$conflict_count = (int) $pdo->query("
SELECT COUNT(*) FROM products_aggregate pa_l
INNER JOIN products_aggregate pa_w
ON pa_w.product_id = $winner_id
AND pa_w.campaign_id = pa_l.campaign_id
AND pa_w.ad_group_id = pa_l.ad_group_id
WHERE pa_l.product_id = $loser_id
")->fetchColumn();
$total_count = (int) $pdo->query("
SELECT COUNT(*) FROM products_aggregate WHERE product_id = $loser_id
")->fetchColumn();
$stat['pa_conflicts_to_delete'] += $conflict_count;
$stat['pa_to_repoint'] += ($total_count - $conflict_count);
if (!$DRY_RUN) {
$pdo->exec("
DELETE pa_l FROM products_aggregate pa_l
INNER JOIN products_aggregate pa_w
ON pa_w.product_id = $winner_id
AND pa_w.campaign_id = pa_l.campaign_id
AND pa_w.ad_group_id = pa_l.ad_group_id
WHERE pa_l.product_id = $loser_id
");
$pdo->exec("UPDATE products_aggregate SET product_id = $winner_id WHERE product_id = $loser_id");
}
// --- products_temp: UNIQUE (product_id, campaign_id, ad_group_id) ---
$pt_conflict = (int) $pdo->query("
SELECT COUNT(*) FROM products_temp pt_l
INNER JOIN products_temp pt_w
ON pt_w.product_id = $winner_id
AND pt_w.campaign_id = pt_l.campaign_id
AND pt_w.ad_group_id = pt_l.ad_group_id
WHERE pt_l.product_id = $loser_id
")->fetchColumn();
$pt_total = (int) $pdo->query("SELECT COUNT(*) FROM products_temp WHERE product_id = $loser_id")->fetchColumn();
$stat['pt_conflicts_to_delete'] += $pt_conflict;
$stat['pt_to_repoint'] += ($pt_total - $pt_conflict);
if (!$DRY_RUN) {
$pdo->exec("
DELETE pt_l FROM products_temp pt_l
INNER JOIN products_temp pt_w
ON pt_w.product_id = $winner_id
AND pt_w.campaign_id = pt_l.campaign_id
AND pt_w.ad_group_id = pt_l.ad_group_id
WHERE pt_l.product_id = $loser_id
");
$pdo->exec("UPDATE products_temp SET product_id = $winner_id WHERE product_id = $loser_id");
}
// --- products_history: UNIQUE (product_id, campaign_id, ad_group_id, date_add) ---
$ph_conflict = (int) $pdo->query("
SELECT COUNT(*) FROM products_history h_l
INNER JOIN products_history h_w
ON h_w.product_id = $winner_id
AND h_w.campaign_id = h_l.campaign_id
AND h_w.ad_group_id = h_l.ad_group_id
AND h_w.date_add = h_l.date_add
WHERE h_l.product_id = $loser_id
")->fetchColumn();
$ph_total = (int) $pdo->query("SELECT COUNT(*) FROM products_history WHERE product_id = $loser_id")->fetchColumn();
$stat['ph_conflicts_to_delete'] += $ph_conflict;
$stat['ph_to_repoint'] += ($ph_total - $ph_conflict);
if (!$DRY_RUN) {
$pdo->exec("
DELETE h_l FROM products_history h_l
INNER JOIN products_history h_w
ON h_w.product_id = $winner_id
AND h_w.campaign_id = h_l.campaign_id
AND h_w.ad_group_id = h_l.ad_group_id
AND h_w.date_add = h_l.date_add
WHERE h_l.product_id = $loser_id
");
$pdo->exec("UPDATE products_history SET product_id = $winner_id WHERE product_id = $loser_id");
}
// --- products_history_30: UNIQUE (product_id, campaign_id, ad_group_id, date_add) ---
$ph30_conflict = (int) $pdo->query("
SELECT COUNT(*) FROM products_history_30 h_l
INNER JOIN products_history_30 h_w
ON h_w.product_id = $winner_id
AND h_w.campaign_id = h_l.campaign_id
AND h_w.ad_group_id = h_l.ad_group_id
AND h_w.date_add = h_l.date_add
WHERE h_l.product_id = $loser_id
")->fetchColumn();
$ph30_total = (int) $pdo->query("SELECT COUNT(*) FROM products_history_30 WHERE product_id = $loser_id")->fetchColumn();
$stat['ph30_conflicts_to_delete'] += $ph30_conflict;
$stat['ph30_to_repoint'] += ($ph30_total - $ph30_conflict);
if (!$DRY_RUN) {
$pdo->exec("
DELETE h_l FROM products_history_30 h_l
INNER JOIN products_history_30 h_w
ON h_w.product_id = $winner_id
AND h_w.campaign_id = h_l.campaign_id
AND h_w.ad_group_id = h_l.ad_group_id
AND h_w.date_add = h_l.date_add
WHERE h_l.product_id = $loser_id
");
$pdo->exec("UPDATE products_history_30 SET product_id = $winner_id WHERE product_id = $loser_id");
}
// --- products_keyword_planner_terms: UNIQUE (product_id, source_url, keyword_text) ---
$kpt_conflict = (int) $pdo->query("
SELECT COUNT(*) FROM products_keyword_planner_terms k_l
INNER JOIN products_keyword_planner_terms k_w
ON k_w.product_id = $winner_id
AND k_w.source_url = k_l.source_url
AND k_w.keyword_text = k_l.keyword_text
WHERE k_l.product_id = $loser_id
")->fetchColumn();
$kpt_total = (int) $pdo->query("SELECT COUNT(*) FROM products_keyword_planner_terms WHERE product_id = $loser_id")->fetchColumn();
$stat['kpt_conflicts_to_delete'] += $kpt_conflict;
$stat['kpt_to_repoint'] += ($kpt_total - $kpt_conflict);
if (!$DRY_RUN) {
$pdo->exec("
DELETE k_l FROM products_keyword_planner_terms k_l
INNER JOIN products_keyword_planner_terms k_w
ON k_w.product_id = $winner_id
AND k_w.source_url = k_l.source_url
AND k_w.keyword_text = k_l.keyword_text
WHERE k_l.product_id = $loser_id
");
$pdo->exec("UPDATE products_keyword_planner_terms SET product_id = $winner_id WHERE product_id = $loser_id");
}
// --- Tabele bez UNIQUE na product_id: prosty UPDATE ---
$stat['comments_to_repoint'] += (int) $pdo->query("SELECT COUNT(*) FROM products_comments WHERE product_id = $loser_id")->fetchColumn();
$stat['alerts_to_repoint'] += (int) $pdo->query("SELECT COUNT(*) FROM campaign_alerts WHERE product_id = $loser_id")->fetchColumn();
$stat['sync_logs_to_repoint'] += (int) $pdo->query("SELECT COUNT(*) FROM products_merchant_sync_log WHERE product_id = $loser_id")->fetchColumn();
if (!$DRY_RUN) {
$pdo->exec("UPDATE products_comments SET product_id = $winner_id WHERE product_id = $loser_id");
$pdo->exec("UPDATE campaign_alerts SET product_id = $winner_id WHERE product_id = $loser_id");
$pdo->exec("UPDATE products_merchant_sync_log SET product_id = $winner_id WHERE product_id = $loser_id");
}
// --- Wreszcie: usun losera z products ---
$stat['products_to_delete']++;
if (!$DRY_RUN) {
$pdo->exec("DELETE FROM products WHERE id = $loser_id");
$pdo->commit();
}
} catch (Throwable $e) {
if (!$DRY_RUN && $pdo->inTransaction()) $pdo->rollBack();
echo "BLAD przy losere $loser_id (winner $winner_id, client $client_id, offer '$oid'): " . $e->getMessage() . "\n";
throw $e;
}
}
$processed++;
if ($processed % 100 === 0) echo " ... przetworzono $processed grup\n";
}
echo "\n" . str_repeat('=', 70) . "\n";
echo " STATYSTYKI " . ($DRY_RUN ? '(DRY-RUN)' : '(WYKONANO)') . "\n";
echo str_repeat('=', 70) . "\n";
foreach ($stat as $k => $v) printf(" %-30s : %d\n", $k, $v);
if ($DRY_RUN) {
echo "\nAby wykonac zmiany, uruchom z flaga --execute\n";
}