{"reference":"AIACT-TD-SUPPLYSCORE-2025","document_date":"2025-03-05","provider":{"name":"Acme Supplies SRL","address":"Rue de la Loi 100, 1040 Brussels, Belgium","contact_email":"ai-compliance@acme-supplies.eu"},"system":{"name":"SupplyScore AI","version":"2.3","intended_purpose":"SupplyScore AI scores new and existing third-party suppliers of Acme Supplies SRL on onboarding risk — financial stability, sanctions exposure, and workforce-compliance indicators — to support procurement decisions. Output is a 0-100 score with a short explanation and is used exclusively to rank suppliers and flag decline-recommended cases for human review; it is never used for fully automated contractual decisions.","high_risk_category":"employment_worker_management","description_of_hardware_software":"Gradient-boosted decision-tree ensemble (LightGBM 4.3) served by a Python 3.12 inference service on Azure Container Apps (4 vCPU, 16 GiB). Input features: 68 tabular attributes. Model artefact pinned by SHA-256 digest in Azure Container Registry. Training runs on a single Azure Standard_NC24ads_A100_v4 VM.","integration_context":"Exposed as a REST endpoint behind Azure API Management and called from the Acme procurement ERP (Dynamics 365 F&O) during the supplier-onboarding workflow. One call per prospective supplier; the score is written back to the supplier master record and displayed to the procurement lead."},"ce_marking":{"applicable":true,"conformity_assessment_route":"harmonised_standard"},"design":{"optimisation_objectives":["Maximise area under the ROC curve for predicting supplier onboarding failure within 12 months","Keep demographic-parity difference across supplier-country clusters below 0.05","Keep mean inference latency under 250 ms at the P95"],"key_design_choices":["Tabular gradient-boosted trees chosen over deep models for interpretability and SHAP-based explanations","Monotonicity constraints on financial-health features (higher liquidity ratio cannot decrease score)","Fairness constraint introduced in v2.3 equalises false-decline rate across EU / non-EU supplier clusters","No use of biometric, special-category (Art. 9 GDPR) or worker personal data — only company-level attributes"],"relevance_of_input_data":"Input features are selected to proxy the three dimensions of supplier onboarding risk: financial stability (8 features from Dun & Bradstreet and publicly filed accounts), sanctions and reputational exposure (12 features from EU, OFAC and UK HMT consolidated lists), and workforce-compliance signals relevant under Annex III §4 of the AI Act (48 features drawn from public labour-inspection registers, social-security good-standing certificates and declared headcount). All features are at the legal-entity level; no individual worker is identified."},"data_governance":{"training_data_summary":"240,000 historical supplier records onboarded by Acme Supplies SRL and peer members of the European Procurement Consortium between 2018-01 and 2024-12, each with a 12-month forward outcome label (failed / did-not-fail). 68 tabular features per record. No natural-person data.","validation_data_summary":"30,000 records sampled from the 2023 calendar year, held out from training, used for hyperparameter tuning and early stopping.","test_data_summary":"30,000 records from 2024-07 to 2024-12, time-held-out, used exclusively for the final metrics reported in section 8.","data_categories":["Company financial ratios (liquidity, leverage, interest coverage)","Sanctions-list matches (EU consolidated, OFAC, UK HMT)","Social-security good-standing indicators","Labour-inspection public findings","Declared headcount and sector code (NACE Rev. 2)"],"bias_mitigation_measures":["Pre-processing: removal of country-of-establishment as a direct feature; retained only as a fairness-monitoring slice","In-training: fairness constraint (demographic-parity difference <= 0.05) across EU / non-EU clusters","Post-hoc: SHAP-based feature-attribution review before each model release by the Responsible AI Committee","Ongoing: quarterly fairness audit across four supplier-country clusters and two firm-size clusters"],"data_provenance":["Dun & Bradstreet Worldbase (licensed, contract ACME-DNB-2024-017)","European Procurement Consortium shared risk dataset (data-use agreement EPC-DUA-v3)","EU consolidated sanctions list (fsd.europa.eu, public, snapshot daily)","National social-security good-standing public registers (BE ONSS/RSZ, FR URSSAF, NL Belastingdienst, DE Deutsche Rentenversicherung)"]},"training":{"methodology":"LightGBM 4.3, 2,000 boosting rounds with early stopping on validation logloss (patience 50). 5-fold time-series cross-validation. Hyperparameter search via Optuna, 120 trials optimising validation AUC subject to the fairness constraint. Class imbalance handled with scale_pos_weight. Monotonicity constraints on 8 financial-health features.","computational_resources":"1 x Azure Standard_NC24ads_A100_v4 (1 x NVIDIA A100 80 GB). Wall-clock training time: 6 h 40 min. Total training energy: approximately 7.2 kWh. The inference model is CPU-only.","training_logs_reference":"s3://acme-aiact-evidence/supplyscore/v2.3/training-logs/ — MLflow run ID 4a2c91e0b7d94a3f8c1d0e6a2f5b3c8e, retained for 10 years after the system is withdrawn from service (Art. 12 AI Act)."},"testing":{"methodology":"Final evaluation on the 30,000-record time-held-out test set (2024-07 to 2024-12). Fairness metrics computed on supplier-country clusters (EU / non-EU) and firm-size clusters (SME / large). Robustness tested against 5% label noise and feature-dropout up to 20%. Explanation stability measured as Spearman correlation between SHAP rankings on paired nearest-neighbour inputs.","metrics_used":[{"metric":"F1 score (failure class)","target":">= 0.80","achieved":"0.87"},{"metric":"AUROC","target":">= 0.85","achieved":"0.91"},{"metric":"Demographic-parity difference (EU vs non-EU)","target":"<= 0.05","achieved":"0.037"},{"metric":"False-decline rate (overall)","target":"<= 0.08","achieved":"0.061"},{"metric":"P95 inference latency","target":"<= 250 ms","achieved":"182 ms"}],"known_limitations":["Training data is concentrated in EU/EEA suppliers (82%); performance on suppliers established in Africa and Latin America is monitored but has wider confidence intervals","Model was not trained on data reflecting a severe macroeconomic downturn (post-2008-style shock) — drift monitoring is required in such conditions","Outputs are decision-support only; the system is not suitable for use as a sole basis for contract termination or legal-effects decisions concerning workers"]},"accuracy_robustness_cybersecurity":{"accuracy_metric_name":"F1 score on the held-out test set (failure class)","accuracy_value":"0.87","robustness_measures":["Input validation schema with range and enum checks on every feature","Graceful degradation: if more than 15% of features are missing for a record, the system returns 'insufficient data' rather than a score","Adversarial-perturbation testing (feature-dropout up to 20%) conducted quarterly","Model rollback runbook: previous version can be re-deployed within 15 minutes"],"cybersecurity_measures":["Inference endpoint behind Azure API Management with OAuth 2.0 and per-client rate limits","TLS 1.3 in transit; AES-256 at rest on Azure Blob Storage for model artefacts and logs","Model artefacts signed and verified by SHA-256 digest at load time","Access to training pipelines gated by Entra ID with MFA and break-glass PIM approval","Annual penetration test by an external CREST-accredited provider; last test 2025-01, no critical findings"]},"human_oversight":{"measures_implemented":["Every prospective supplier with a decline-recommended score (<= 40) is reviewed by a designated procurement lead before any contractual consequence","Procurement leads receive mandatory AI-literacy training (Art. 4 AI Act) and specific SupplyScore decision-support training before gaining access","The user interface displays the top 5 SHAP feature contributions alongside every score and a prominent 'override with reason' button","A stop-button disables scoring for the full workflow within 5 minutes; the procurement function can operate on legacy heuristics while the system is disabled","Automated scoring is explicitly disabled for supplier onboarding decisions that would result in termination of an existing contract affecting workers"]},"risk_management":{"reference_to_system":"AIACT-RMS-SUPPLYSCORE-2025 — Art. 9 risk management file, stored at s3://acme-aiact-evidence/supplyscore/v2.3/risk-management/, reviewed quarterly by the Responsible AI Committee."},"monitoring_plan":{"post_market_monitoring":"Quarterly drift audit (PSI on each feature, per-cluster AUROC and F1 on a rolling 90-day evaluation set), quarterly fairness audit across the four supplier-country clusters and two firm-size clusters, monthly SLO review (latency, error rate, rate-limit saturation). Material deviations trigger the incident process; substantial modifications trigger a new conformity assessment under Art. 43(4).","incident_reporting_contact":"ai-incidents@acme-supplies.eu"},"changes":{"version_history":[{"version":"1.0","date":"2024-06-15","change_summary":"Initial release. Internal pilot on 3 procurement categories; decision-support only."},{"version":"2.0","date":"2024-12-10","change_summary":"Full retraining on 2018-2024 data (240,000 records) with expanded feature set (48 -> 68 features) including sanctions-list signals and social-security good-standing indicators."},{"version":"2.3","date":"2025-02-20","change_summary":"Added fairness constraint on demographic-parity difference across EU / non-EU supplier clusters (<= 0.05). Minor update to monotonicity constraints on leverage features. No change to the intended purpose or to the Annex III classification."}]},"compliance_declaration_reference":"AIACT-DOC-SUPPLYSCORE-2025 — EU declaration of conformity drawn up under Art. 47 AI Act, signed by the CEO on 2025-03-01, kept at Acme Supplies SRL registered office for 10 years after the system is placed on the market."}