Refactor: Remove Azure dependencies, use local storage
Replaced Azure Blob Storage and Storage Queue with local temp folders and an in-memory queue for file storage and async processing. Updated `IFileStorage` and `IJobQueue` interfaces to support the new architecture. Modified `TenantSettings` and `ApplyStampHandler` to use local file paths. Updated `JobProcessorService` to handle in-memory queue jobs. Added file cleanup policies to `LocalFileStorage`. Revised roadmap and documentation to reflect the shift to local-first architecture, emphasizing simplicity, reduced cloud dependencies, and single-server readiness. Logging now uses file-based storage instead of Application Insights. Adjusted production deployment and health check phases to align with the new approach.
This commit is contained in:
399
DocumentOperator.API/PROJECT_STATUS.md
Normal file
399
DocumentOperator.API/PROJECT_STATUS.md
Normal file
@@ -0,0 +1,399 @@
|
||||
# DocumentOperator - Projekt Status & Zeitplan
|
||||
|
||||
> **Letzte Aktualisierung:** 22.06.2026
|
||||
> **Aktueller Status:** Phase 3 - Infrastructure Layer (in Bearbeitung)
|
||||
> **Gesamtfortschritt:** ~25% (2.5 von 11 Phasen abgeschlossen)
|
||||
|
||||
---
|
||||
|
||||
## ?? Projekt-Übersicht
|
||||
|
||||
**DocumentOperator** ist ein zentralisierter REST API Service für PDF-Dokumenten-Operationen in einer Multi-Tenant DMS-Umgebung.
|
||||
|
||||
### Kern-Features
|
||||
- ? PDF Validierung (Metadaten-Extraktion)
|
||||
- ? Attachment-Extraktion
|
||||
- ? PDF-Konkatenation (asynchron bei großen Dateien)
|
||||
- ? Stempel/Wasserzeichen (tenant-spezifisch)
|
||||
- ? Zertifikat-Einbettung
|
||||
|
||||
### Technologie-Stack
|
||||
- **.NET 8** (ASP.NET Core Minimal APIs)
|
||||
- **Clean Architecture** (pragmatisch, nicht dogmatisch)
|
||||
- **DevExpress PDF Core** (Universal License verfügbar)
|
||||
- **MediatR** (CQRS Pattern)
|
||||
- **EF Core + SQLite** (Tenant-Management)
|
||||
- **Lokale Temp-Ordner** (File Storage mit IFileStorage Abstraction)
|
||||
- **In-Memory Queue** (Async Processing)
|
||||
- **Polly** (Resilience: Retry, Circuit Breaker)
|
||||
- **Redis** (Caching, Rate-Limiting)
|
||||
- **Serilog** (Structured Logging)
|
||||
|
||||
---
|
||||
|
||||
## ?? Phasen-Übersicht
|
||||
|
||||
| Phase | Name | Status | Fortschritt | Zeitaufwand |
|
||||
|-------|------|--------|-------------|-------------|
|
||||
| **1** | Foundation | ? Abgeschlossen | 100% | ~2 Tage |
|
||||
| **2** | Domain Layer | ? Abgeschlossen | 100% | ~1 Tag |
|
||||
| **3** | Infrastructure Layer | ?? In Bearbeitung | 50% | ~3-4 Tage |
|
||||
| **4** | Application Layer | ? Ausstehend | 0% | ~2-3 Tage |
|
||||
| **5** | API Layer | ? Ausstehend | 0% | ~2 Tage |
|
||||
| **5.5** | Health Checks & Resilience | ? Ausstehend | 0% | ~1 Tag |
|
||||
| **6** | Weitere Features (synchron) | ? Ausstehend | 0% | ~3-4 Tage |
|
||||
| **6.5** | Async Processing (In-Memory Queue) | ? Ausstehend | 0% | ~2 Tage |
|
||||
| **7** | Swagger & API Docs | ? Ausstehend | 0% | ~1 Tag |
|
||||
| **8** | Multi-Tenancy (DB-based) | ? Ausstehend | 0% | ~3-4 Tage |
|
||||
| **9** | File Storage (Lokale Temp-Ordner) | ? Ausstehend | 0% | ~1 Tag |
|
||||
| **10** | Logging & Monitoring | ? Ausstehend | 0% | ~2 Tage |
|
||||
| **11** | Production Deployment | ? Ausstehend | 0% | ~2-3 Tage |
|
||||
|
||||
**Gesamtaufwand (Schätzung):** ~24-30 Arbeitstage (5-6 Wochen bei 1 Entwickler)
|
||||
|
||||
---
|
||||
|
||||
## ? Bereits erledigt (Phasen 1-2.5)
|
||||
|
||||
### Phase 1: Foundation ?
|
||||
- [x] Solution-Struktur erstellt (4 Projekte: API, Application, Infrastructure, Domain)
|
||||
- [x] Clean Architecture Dependencies korrekt (Dependency Rule eingehalten)
|
||||
- [x] NuGet Packages installiert (MediatR, FluentValidation, DevExpress, xUnit, Moq, FluentAssertions)
|
||||
- [x] Folder-Struktur nach Vertical Slice Architecture
|
||||
- [x] Serilog Setup (Structured Logging)
|
||||
- [x] appsettings.json Konfiguration
|
||||
|
||||
### Phase 2: Domain Layer ?
|
||||
- [x] Domain Exceptions (4 Stück: DomainException, DomainValidationException, NotFoundException, PdfProcessingException)
|
||||
- [x] Enums (DocumentOperationType, ProcessingStatus)
|
||||
- [x] Value Objects (Base64String, TenantId, PdfMetadata) - selbst-validierend, immutable
|
||||
|
||||
### Phase 3: Infrastructure Layer (50% abgeschlossen) ??
|
||||
- [x] IPdfProcessor Interface (Application/Common/Interfaces/)
|
||||
- [x] Test-Struktur erstellt (Unit/Infrastructure/Services/PdfProcessing/)
|
||||
- [x] Test-PDF Datei (valid.pdf als Embedded Resource)
|
||||
- [x] DevExpressPdfProcessorTests.cs (TDD Red Phase - 6 Tests geschrieben)
|
||||
- [ ] **IN ARBEIT:** DevExpressPdfProcessor.cs implementieren (TDD Green Phase)
|
||||
- [ ] **Ausstehend:** Polly Resilience Integration (Retry, Circuit Breaker, Timeout)
|
||||
- [ ] **Ausstehend:** IFileStorage Interface + LocalFileStorage (Dev)
|
||||
|
||||
---
|
||||
|
||||
## ?? Aktuell in Bearbeitung (Phase 3)
|
||||
|
||||
### Step 3.2: DevExpressPdfProcessor implementieren (TDD)
|
||||
**Was wurde gemacht:**
|
||||
- ? Tests geschrieben (Red Phase)
|
||||
- ValidateAsync_ValidPdf_ReturnsMetadata
|
||||
- ValidateAsync_InvalidPdf_ThrowsPdfProcessingException
|
||||
- ValidateAsync_CorruptedPdf_ThrowsPdfProcessingException
|
||||
- ValidateAsync_EmptyByteArray_ThrowsDomainValidationException
|
||||
- ValidateAsync_NullByteArray_ThrowsDomainValidationException
|
||||
- ValidateAsync_PdfWithAttachments_ReturnsCorrectAttachmentCount
|
||||
|
||||
**Was steht an:**
|
||||
- ? DevExpressPdfProcessor.cs implementieren (Green Phase)
|
||||
- DevExpress PDF API Integration
|
||||
- PdfDocumentProcessor verwenden
|
||||
- Exception Handling
|
||||
- Metadaten extrahieren (Seitenzahl, Dateigröße, Anhänge)
|
||||
|
||||
**Zeitaufwand:** ~1-2 Tage
|
||||
|
||||
---
|
||||
|
||||
## ? Nächste Schritte (Phase 4-5)
|
||||
|
||||
### Phase 4: Application Layer (~2-3 Tage)
|
||||
**Priorität:** Hoch
|
||||
**Abhängigkeiten:** Phase 3 muss abgeschlossen sein
|
||||
|
||||
**Tasks:**
|
||||
- [ ] MediatR Setup (DependencyInjection.cs, ValidationBehavior.cs, LoggingBehavior.cs)
|
||||
- [ ] ValidatePdf Feature (Query, Handler, Validator)
|
||||
- [ ] DTOs (ValidatePdfRequest, ValidatePdfResponse)
|
||||
- [ ] Unit Tests für Handler
|
||||
|
||||
**Deliverable:** Erste komplette Feature-Implementierung (ValidatePdf)
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: API Layer (~2 Tage)
|
||||
**Priorität:** Hoch
|
||||
**Abhängigkeiten:** Phase 4 muss abgeschlossen sein
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Exception Handling Middleware (zentrale Exception ? HTTP Status Code Mapping)
|
||||
- [ ] Minimal API Endpoint (POST /api/v1/documents/validate)
|
||||
- [ ] Swagger Configuration (API-Key Header Support)
|
||||
- [ ] Integration Test (End-to-End)
|
||||
|
||||
**Deliverable:** Funktionierender REST API Endpoint (testbar via Swagger)
|
||||
|
||||
---
|
||||
|
||||
## ?? Kritische Phasen (Production-Ready)
|
||||
|
||||
### Phase 5.5: Health Checks & Resilience (~1 Tag)
|
||||
**Priorität:** Mittel (aber wichtig für Production!)
|
||||
**Warum früh?** Load Balancer / Kubernetes brauchen Health Checks
|
||||
|
||||
**Tasks:**
|
||||
- [ ] DevExpressPdfHealthCheck (Smoke Test)
|
||||
- [ ] /health Endpoint (Liveness/Readiness Probes)
|
||||
- [ ] Polly Resilience (Retry, Circuit Breaker, Timeout für DevExpress Calls)
|
||||
|
||||
**Deliverable:** Production-ready Health Checks
|
||||
|
||||
---
|
||||
|
||||
### Phase 6.5: Async Processing (~2 Tage)
|
||||
**Priorität:** Mittel
|
||||
**Warum wichtig?** PDF-Konkatenation von 50 PDFs = 10+ Sekunden (HTTP Timeout!)
|
||||
|
||||
**Tasks:**
|
||||
- [ ] In-Memory Queue Setup (IJobQueue Interface)
|
||||
- [ ] Background Worker (JobProcessorService - IHostedService)
|
||||
- [ ] Async Endpoints (POST ? JobId, GET /jobs/{id} ? Status)
|
||||
- [ ] Job Status Tracking (In-Memory Dictionary mit ConcurrentDictionary)
|
||||
|
||||
**Deliverable:** Queue-basiertes Async Processing (In-Memory, einfache Implementierung)
|
||||
|
||||
**Hinweis:** In-Memory Queue = nicht persistent (Server-Neustart löscht Jobs). Für Production mit Load Balancer später auf Redis Queue upgraden.
|
||||
|
||||
---
|
||||
|
||||
### Phase 8: Multi-Tenancy (DB-based) (~3-4 Tage)
|
||||
**Priorität:** Hoch (für Multi-Tenant DMS!)
|
||||
**Warum DB-based?** Skalierbar, API-Key Rotation, Audit-Log
|
||||
|
||||
**Tasks:**
|
||||
- [ ] EF Core + SQLite Setup (Tenant-Datenbank)
|
||||
- [ ] Tenant/TenantSettings Entities
|
||||
- [ ] TenantRepository (ITenantRepository)
|
||||
- [ ] TenantResolutionMiddleware (API-Key ? Tenant, Redis Cache)
|
||||
- [ ] Tenant Admin API (CRUD für API-Keys)
|
||||
- [ ] API-Key Hashing (BCrypt)
|
||||
|
||||
**Deliverable:** Multi-Tenant fähige API mit DB-basiertem Tenant-Management
|
||||
|
||||
---
|
||||
|
||||
### Phase 9: File Storage (Lokale Temp-Ordner) (~1 Tag)
|
||||
**Priorität:** Mittel
|
||||
**Warum wichtig?** Temporäre Dateien für verarbeitete PDFs (Download-Links)
|
||||
|
||||
**Tasks:**
|
||||
- [ ] IFileStorage Interface
|
||||
- [ ] LocalFileStorage Implementation (appsettings: TempFolder-Pfad)
|
||||
- [ ] TempFileCleanupService (IHostedService - täglich um 3 Uhr)
|
||||
- [ ] Optionale Retention Policy (z.B. 24 Stunden)
|
||||
|
||||
**Deliverable:** Lokales File Storage mit automatischem Cleanup
|
||||
|
||||
**Hinweis:** Single-Server Setup (einfach). Für Multi-Server später auf Shared Network Drive (UNC-Pfad) upgraden.
|
||||
|
||||
---
|
||||
|
||||
## ?? Risiken & Abhängigkeiten
|
||||
|
||||
### Technische Risiken
|
||||
1. **DevExpress PDF API Stabilität**
|
||||
- **Risiko:** Unbekannte Edge Cases (korrupte PDFs, spezielles PDF-Format)
|
||||
- **Mitigation:** Polly Resilience (Retry, Circuit Breaker), umfangreiche Tests
|
||||
- **Impact:** Mittel
|
||||
|
||||
2. **In-Memory Queue (nicht persistent)**
|
||||
- **Risiko:** Server-Neustart löscht alle laufenden Jobs
|
||||
- **Mitigation:** Single-Server Setup empfohlen, später Redis Queue für Production
|
||||
- **Impact:** Niedrig (für MVP akzeptabel)
|
||||
|
||||
3. **Multi-Tenancy Komplexität**
|
||||
- **Risiko:** Tenant-Isolation muss 100% korrekt sein (Security!)
|
||||
- **Mitigation:** TDD, umfangreiche Integration Tests, Code Reviews
|
||||
- **Impact:** Hoch (aber planbar)
|
||||
|
||||
### Externe Abhängigkeiten
|
||||
- ? **DevExpress Universal License:** Verfügbar
|
||||
- ? **Redis Server:** Optional für Phase 8 (Tenant-Caching), Phase 11 (Rate-Limiting)
|
||||
- Alternative: In-Memory Cache für Single-Server Setup
|
||||
- ? **Shared Network Drive:** Nur bei Multi-Server Setup (für File Storage)
|
||||
- Alternative: Lokale Temp-Ordner für Single-Server
|
||||
|
||||
---
|
||||
|
||||
## ?? Zeitplan & Milestones
|
||||
|
||||
### Milestone 1: MVP (Minimum Viable Product) ??
|
||||
**Ziel:** Funktionierender API Endpoint (ValidatePdf)
|
||||
**Phasen:** 1-5 abgeschlossen
|
||||
**Zeitaufwand:** ~10-12 Arbeitstage
|
||||
**ETA:** Ende KW 4 / Anfang KW 5 (bei 1 Entwickler)
|
||||
|
||||
**Deliverables:**
|
||||
- ? REST API Endpoint: POST /api/v1/documents/validate
|
||||
- ? Swagger UI (testbar)
|
||||
- ? Exception Handling
|
||||
- ? Unit Tests + Integration Tests
|
||||
|
||||
---
|
||||
|
||||
### Milestone 2: Production-Ready (Core Features) ??
|
||||
**Ziel:** Production-ready API mit Health Checks, Resilience, Multi-Tenancy
|
||||
**Phasen:** 1-5.5, 6 (nur ValidatePdf + ExtractAttachments), 7, 8 abgeschlossen
|
||||
**Zeitaufwand:** ~20-25 Arbeitstage
|
||||
**ETA:** Ende KW 7 / Anfang KW 8
|
||||
|
||||
**Deliverables:**
|
||||
- ? Alle CRUD-Operationen (ValidatePdf, ExtractAttachments)
|
||||
- ? Multi-Tenancy (DB-based, API-Key Management)
|
||||
- ? Health Checks (/health Endpoint)
|
||||
- ? Resilience (Polly)
|
||||
- ? Swagger Documentation
|
||||
- ? Production-ready Logging (Correlation IDs)
|
||||
|
||||
---
|
||||
|
||||
### Milestone 3: Full Feature Set ??
|
||||
**Ziel:** Alle Features + Async Processing + File Storage
|
||||
**Phasen:** Alle 11 Phasen abgeschlossen
|
||||
**Zeitaufwand:** ~24-30 Arbeitstage
|
||||
**ETA:** Ende KW 8 / Anfang KW 9
|
||||
|
||||
**Deliverables:**
|
||||
- ? Alle PDF-Operationen (ValidatePdf, ExtractAttachments, ConcatenatePdfs, ApplyStamp, EmbedCertificate)
|
||||
- ? Async Processing (In-Memory Queue für große Operationen)
|
||||
- ? Lokale Temp-Ordner (File Storage mit automatischem Cleanup)
|
||||
- ? Logging & Monitoring (Seq, File Logging)
|
||||
- ? IIS Deployment-ready
|
||||
- ? Rate-Limiting (Redis-based)
|
||||
|
||||
---
|
||||
|
||||
## ?? Empfehlung für Projekt-Planung
|
||||
|
||||
### Option 1: Agile Iterationen (Empfohlen ?)
|
||||
**Strategie:** 3 Milestones iterativ ausrollen
|
||||
|
||||
**Vorteile:**
|
||||
- ? Frühes Feedback (MVP nach 2 Wochen testbar)
|
||||
- ? Risiken früh erkannt (DevExpress Integration in Milestone 1)
|
||||
- ? Flexibel (Anforderungen können sich ändern)
|
||||
|
||||
**Zeitplan:**
|
||||
- **Sprint 1 (2 Wochen):** Milestone 1 (MVP) - Phase 1-5
|
||||
- **Sprint 2 (3 Wochen):** Milestone 2 (Production-Ready) - Phase 5.5, 6, 7, 8
|
||||
- **Sprint 3 (1-2 Wochen):** Milestone 3 (Full Feature Set) - Phase 6.5, 9, 10, 11
|
||||
|
||||
**Gesamtdauer:** ~6 Wochen
|
||||
|
||||
---
|
||||
|
||||
### Option 2: Wasserfall (Nicht empfohlen ??)
|
||||
**Strategie:** Alle Phasen komplett abarbeiten, dann erst testen
|
||||
|
||||
**Nachteile:**
|
||||
- ? Spätes Feedback (erst nach 6 Wochen testbar)
|
||||
- ? Risiken spät erkannt
|
||||
- ? Keine Flexibilität
|
||||
|
||||
**Gesamtdauer:** ~6-7 Wochen (gleich, aber höheres Risiko)
|
||||
|
||||
---
|
||||
|
||||
## ?? Ressourcen-Bedarf
|
||||
|
||||
### Entwickler
|
||||
- **Aktuell:** 1 Entwickler (außerhalb der Kernarbeitszeit)
|
||||
- **Empfehlung:** 1 Entwickler ausreichend (bei agilen Sprints)
|
||||
- **Alternative:** 2 Entwickler = Halbierung der Zeit (~3 Wochen statt 6)
|
||||
|
||||
### Infrastruktur (Production)
|
||||
- **Lokaler File Server** (für Temp-Ordner) - ODER Shared Network Drive bei Multi-Server
|
||||
- **Redis Server** (für Tenant-Caching, Rate-Limiting)
|
||||
- **SQL Server** (optional, aktuell SQLite) - erst bei > 10.000 Requests/Sekunde
|
||||
|
||||
### Kosten-Schätzung
|
||||
- **Redis Cache (Basic):** ~15 EUR/Monat (optional: In-Memory Cache für Single-Server)
|
||||
- **Seq (Self-Hosted):** Kostenlos (oder Seq Cloud: ~20 EUR/Monat)
|
||||
- **File Storage:** Kostenlos (lokale Festplatte)
|
||||
|
||||
**Gesamt:** ~0-35 EUR/Monat (abhängig von Redis + Seq Cloud)
|
||||
|
||||
---
|
||||
|
||||
## ? Nächste Schritte (diese Woche)
|
||||
|
||||
### Priorität 1: Phase 3 abschließen
|
||||
- [ ] DevExpressPdfProcessor.cs implementieren (TDD Green Phase)
|
||||
- [ ] Alle Tests grün machen
|
||||
- [ ] Code Review
|
||||
|
||||
**Zeitaufwand:** 1-2 Tage
|
||||
**ETA:** Dienstag/Mittwoch
|
||||
|
||||
---
|
||||
|
||||
### Priorität 2: Phase 4 starten (Application Layer)
|
||||
- [ ] MediatR Setup
|
||||
- [ ] ValidatePdf Feature (Query, Handler, Validator)
|
||||
|
||||
**Zeitaufwand:** 2-3 Tage
|
||||
**ETA:** Ende der Woche (Freitag)
|
||||
|
||||
---
|
||||
|
||||
### Priorität 3: Milestone 1 erreichen (MVP)
|
||||
- [ ] Phase 5 (API Layer)
|
||||
- [ ] Erster testbarer Endpoint via Swagger
|
||||
|
||||
**Zeitaufwand:** 2 Tage
|
||||
**ETA:** Anfang nächste Woche (Montag/Dienstag)
|
||||
|
||||
---
|
||||
|
||||
## ?? Fragen / Entscheidungen erforderlich
|
||||
|
||||
1. **Redis Server:**
|
||||
Ist Redis-Instanz verfügbar? (Benötigt für Phase 8 - Tenant-Caching, Phase 11 - Rate-Limiting)
|
||||
**Alternative:** In-Memory Cache für Dev, Redis erst für Production
|
||||
|
||||
2. **Deployment-Ziel:**
|
||||
IIS? Docker/Kubernetes?
|
||||
**Impact:** Beeinflusst Phase 11 (Production Deployment)
|
||||
**Empfehlung:** IIS (Windows Server) - einfachste Lösung für .NET 8 APIs
|
||||
|
||||
3. **Priorität Async Processing (Phase 6.5):**
|
||||
Ist In-Memory Queue-basiertes Async Processing Pflicht oder Nice-to-have?
|
||||
**Impact:** Kann Zeit sparen (~2 Tage), wenn nicht benötigt
|
||||
**Hinweis:** In-Memory Queue = nicht persistent (Server-Neustart löscht Jobs)
|
||||
|
||||
4. **Multi-Server Setup:**
|
||||
Wird Load Balancer mit mehreren API-Instanzen benötigt?
|
||||
**Impact:** Multi-Server = Shared Network Drive statt lokale Temp-Ordner notwendig
|
||||
**Empfehlung:** Single-Server Setup vorerst (einfacher)
|
||||
|
||||
---
|
||||
|
||||
## ?? Fazit & Empfehlung
|
||||
|
||||
**Projekt-Status:** ? Auf gutem Weg (25% abgeschlossen)
|
||||
**Architektur:** ? Solide (Clean Architecture, TDD, Vertical Slices)
|
||||
**Technologie-Stack:** ? Modern (.NET 8, Minimal APIs, Polly, EF Core) - **OHNE Azure-Abhängigkeiten**
|
||||
**Risiken:** ?? Gering (mit Polly Resilience abgefedert)
|
||||
|
||||
**Empfehlung:**
|
||||
1. ? **Phase 3 abschließen** (diese Woche)
|
||||
2. ? **Milestone 1 (MVP) erreichen** (nächste Woche)
|
||||
3. ? **Agile Iterationen** nutzen (3 Sprints à 2-3 Wochen)
|
||||
4. ? **Frühes Feedback** einholen (nach Milestone 1)
|
||||
|
||||
**ETA für Production-Ready (Milestone 2):** Ende KW 7 / Anfang KW 8 (bei 1 Entwickler)
|
||||
|
||||
**Änderung (22.06.2026):** Azure Services (Blob Storage, Storage Queue) vollständig entfernt - stattdessen lokale Temp-Ordner + In-Memory Queue (einfacher, keine Cloud-Abhängigkeiten). Alle Azure-Referenzen in der Dokumentation bereinigt.
|
||||
|
||||
---
|
||||
|
||||
**Erstellt von:** DocumentOperator Entwicklungsteam
|
||||
**Datum:** 22.06.2026
|
||||
**Version:** 1.3 (Azure-Referenzen vollständig entfernt)
|
||||
@@ -1,6 +1,6 @@
|
||||
# ?? DocumentOperator - Project Roadmap (Pragmatic Edition)
|
||||
|
||||
> **Last Updated:** 17.01.2025 (Updated with Resilience, Async Processing, Multi-Tenancy DB) | **Status:** In Development | **Phase:** 3 (Infrastructure Layer)
|
||||
> **Last Updated:** 22.06.2026 (Azure-Referenzen vollständig entfernt) | **Status:** In Development | **Phase:** 3 (Infrastructure Layer)
|
||||
|
||||
---
|
||||
|
||||
@@ -9,22 +9,22 @@
|
||||
**Was ist neu in diesem Update?**
|
||||
|
||||
1. **? Multi-Tenancy:** Database-based (EF Core + SQLite + Redis Cache) statt appsettings.json
|
||||
2. **? Async Processing:** Queue-based (Azure Storage Queue + Background Worker) für große Operationen
|
||||
3. **? File Storage:** Azure Blob Storage + IFileStorage Abstraction (Multi-Server fähig!)
|
||||
2. **? Async Processing:** In-Memory Queue-based + Background Worker für große Operationen
|
||||
3. **? File Storage:** Lokale Temp-Ordner mit IFileStorage Abstraction
|
||||
4. **? Resilience:** Polly (Retry, Circuit Breaker, Timeout) für DevExpress Calls
|
||||
5. **?? Health Checks:** FRÜH implementieren (Phase 5.5 statt Phase 9)
|
||||
6. **? Logging:** Correlation IDs + Seq + Application Insights
|
||||
7. **? 11 neue NuGet Packages:** EF Core, Polly, Azure Storage, BCrypt, Seq, Correlation IDs
|
||||
6. **? Logging:** Correlation IDs + Seq + File Logging
|
||||
7. **? 9 neue NuGet Packages:** EF Core, Polly, BCrypt, Seq, Correlation IDs
|
||||
8. **? 6 neue Phasen:** 5.5, 6.5, 8, 9, 10, 11 (insgesamt 11 Phasen statt 9)
|
||||
9. **? 11 Key Learnings:** Dokumentiert (statt 5)
|
||||
10. **? Technology Stack:** Komplett aktualisiert mit allen neuen Dependencies
|
||||
|
||||
**Warum diese Änderungen?**
|
||||
- **Skalierbarkeit:** Multi-Server Support (Azure Blob, Redis Cache)
|
||||
- **Einfachheit:** Lokale Temp-Ordner (keine Cloud-Abhängigkeiten)
|
||||
- **Security:** API-Key Hashing (BCrypt), Rotation möglich
|
||||
- **Performance:** Async Processing für große Operationen (keine HTTP Timeouts)
|
||||
- **Resilience:** Production-ready (Polly Retry/Circuit Breaker)
|
||||
- **Monitoring:** Correlation IDs, Seq, Application Insights
|
||||
- **Monitoring:** Correlation IDs, Seq, File Logging
|
||||
- **Wartbarkeit:** Clean Architecture bleibt pragmatisch, aber production-ready!
|
||||
|
||||
---
|
||||
@@ -120,7 +120,7 @@ Client Application
|
||||
?
|
||||
[Returns: { "jobId": "abc123", "status": "Pending" }] - Sofort
|
||||
?
|
||||
Background Worker (IHostedService) ? Azure Storage Queue ? Verarbeitung
|
||||
Background Worker (IHostedService) ? In-Memory Queue ? Verarbeitung
|
||||
?
|
||||
Client Poll: [GET /api/v1/jobs/abc123] ? { "status": "Processing", "progress": 45% }
|
||||
?
|
||||
@@ -133,7 +133,7 @@ Client Poll: [GET /api/v1/jobs/abc123] ? { "status": "Success", "resultUrl": "/d
|
||||
3. API validiert Input (FluentValidation in MediatR Pipeline)
|
||||
4. Handler konvertiert PDF ? Byte-Array
|
||||
5. DevExpress Service führt Operation durch (mit Polly Retry/Circuit Breaker)
|
||||
6. Ergebnis wird in Azure Blob Storage gespeichert (Multi-Server!)
|
||||
6. Ergebnis wird in lokalem Temp-Ordner gespeichert
|
||||
7. Ergebnis wird als Base64 zurückgegeben (oder Download-Link)
|
||||
|
||||
---
|
||||
@@ -455,8 +455,8 @@ public class TenantSettings
|
||||
{
|
||||
public Guid Id { get; set; }
|
||||
public Guid TenantId { get; set; }
|
||||
public string LogoBlobPath { get; set; } // Azure Blob Storage Pfad
|
||||
public string CertificateBlobPath { get; set; } // Azure Blob Storage Pfad
|
||||
public string LogoFilePath { get; set; } // Lokaler Dateipfad (z.B. "logos/tenant-a/stamp.png")
|
||||
public string CertificateFilePath { get; set; } // Lokaler Dateipfad (z.B. "certs/tenant-a/cert.pfx")
|
||||
public int RateLimitPerMinute { get; set; } // Rate-Limiting
|
||||
|
||||
// Navigation
|
||||
@@ -469,14 +469,14 @@ public class TenantSettings
|
||||
public class ApplyStampHandler : IRequestHandler<ApplyStampCommand, byte[]>
|
||||
{
|
||||
private readonly ITenantContext _tenantContext;
|
||||
private readonly IFileStorage _fileStorage; // Azure Blob Storage
|
||||
private readonly IFileStorage _fileStorage; // Lokaler File Storage
|
||||
|
||||
public async Task<byte[]> Handle(ApplyStampCommand command, CancellationToken ct)
|
||||
{
|
||||
// Tenant-spezifisches Logo aus DB Settings laden
|
||||
var logoPath = _tenantContext.CurrentTenant.Settings.LogoBlobPath;
|
||||
var logoPath = _tenantContext.CurrentTenant.Settings.LogoFilePath;
|
||||
|
||||
// Logo aus Azure Blob Storage laden
|
||||
// Logo aus lokalem Dateisystem laden
|
||||
var logoBytes = await _fileStorage.GetAsync(logoPath);
|
||||
|
||||
// Stamp mit Logo anwenden
|
||||
@@ -537,7 +537,6 @@ public class ApplyStampHandler : IRequestHandler<ApplyStampCommand, byte[]>
|
||||
| **Microsoft.EntityFrameworkCore** | 8.0.0 | **NEU:** ORM für Tenant-Datenbank |
|
||||
| **Microsoft.EntityFrameworkCore.Sqlite** | 8.0.0 | **NEU:** SQLite Provider (Tenant-DB) |
|
||||
| **Microsoft.EntityFrameworkCore.Tools** | 8.0.0 | **NEU:** Migrations-Support |
|
||||
| **Azure.Storage.Blobs** | 12.22.3 | **NEU:** Azure Blob Storage (File Storage) |
|
||||
| **Polly** | 8.5.0 | **NEU:** Resilience (Retry, Circuit Breaker, Timeout) |
|
||||
| **BCrypt.Net-Next** | 4.0.3 | **NEU:** API-Key Hashing (Security) |
|
||||
| **Microsoft.Extensions.Options.ConfigurationExtensions** | 8.0.0 | Options Pattern |
|
||||
@@ -552,10 +551,9 @@ public class ApplyStampHandler : IRequestHandler<ApplyStampCommand, byte[]>
|
||||
|
||||
**Warum diese neuen Pakete?**
|
||||
- **EF Core + SQLite:** Tenant-Datenbank (API-Keys, Settings) - skalierbar ohne SQL Server
|
||||
- **Azure Blob Storage:** Multi-Server File Storage (Logos, Zertifikate, Temp-Files)
|
||||
- **Polly:** Resilience für DevExpress Calls (Retry bei Transient Errors, Circuit Breaker)
|
||||
- **BCrypt:** Sichere API-Key Hashes (NICHT Klartext in DB!)
|
||||
- **Seq:** Log-Browsing UI für Development (Production: Application Insights)
|
||||
- **Seq:** Log-Browsing UI für Development
|
||||
- **Correlation IDs:** Request-Tracking über alle Logs (Debugging leichter)
|
||||
|
||||
#### Domain Layer
|
||||
@@ -664,7 +662,7 @@ DocumentOperator.Application/
|
||||
? ??? Interfaces/ ? Abstractions für Infrastructure
|
||||
? ? ??? IPdfProcessor.cs
|
||||
? ? ??? IFileStorage.cs ? **NEU:** File Storage Abstraction
|
||||
? ? ??? IJobQueue.cs ? **NEU:** Queue Abstraction (Azure Storage Queue)
|
||||
? ? ??? IJobQueue.cs ? **NEU:** Queue Abstraction (In-Memory Queue)
|
||||
? ? ??? ITenantRepository.cs ? **NEU:** Tenant-DB Abstraction
|
||||
? ??? Behaviors/ ? MediatR Pipeline Behaviors
|
||||
? ? ??? ValidationBehavior.cs ? FluentValidation Integration
|
||||
@@ -715,10 +713,9 @@ DocumentOperator.Infrastructure/
|
||||
? ??? PdfProcessing/
|
||||
? ? ??? DevExpressPdfProcessor.cs ? IPdfProcessor Implementation (mit Polly Resilience)
|
||||
? ??? FileStorage/
|
||||
? ? ??? AzureBlobFileStorage.cs ? **NEU:** IFileStorage Implementation (Azure Blob)
|
||||
? ? ??? LocalFileStorage.cs ? **NEU:** IFileStorage Implementation (Dev/Test)
|
||||
? ? ??? LocalFileStorage.cs ? **NEU:** IFileStorage Implementation (lokaler Temp-Ordner)
|
||||
? ??? Queue/
|
||||
? ??? AzureStorageJobQueue.cs ? **NEU:** IJobQueue Implementation (Azure Storage Queue)
|
||||
? ??? InMemoryJobQueue.cs ? **NEU:** IJobQueue Implementation (In-Memory Queue)
|
||||
??? Data/
|
||||
? ??? TenantDbContext.cs ? **NEU:** EF Core DbContext (Tenant-DB)
|
||||
? ??? Entities/
|
||||
@@ -732,20 +729,18 @@ DocumentOperator.Infrastructure/
|
||||
? ??? TempFileCleanupService.cs ? **NEU:** IHostedService für Temp-File Cleanup (täglich)
|
||||
??? Configuration/
|
||||
? ??? DocumentOperatorSettings.cs ? Options Pattern Class
|
||||
? ??? AzureBlobSettings.cs ? **NEU:** Azure Blob Storage Configuration
|
||||
? ??? AzureQueueSettings.cs ? **NEU:** Azure Storage Queue Configuration
|
||||
? ??? FileStorageSettings.cs ? **NEU:** File Storage Configuration (Temp-Ordner Pfad)
|
||||
? ??? RedisSettings.cs ? **NEU:** Redis Cache Configuration
|
||||
??? DependencyInjection.cs ? Service Registration
|
||||
```
|
||||
|
||||
**Was gehört hierher:**
|
||||
- ? DevExpress Integration (mit Polly Resilience!)
|
||||
- ? **File Storage:** Azure Blob Storage + Local File Storage (Abstraction!)
|
||||
- ? **Queue:** Azure Storage Queue für Async Processing
|
||||
- ? **File Storage:** Lokaler Temp-Ordner (IFileStorage Abstraction)
|
||||
- ? **Queue:** In-Memory Queue für Async Processing
|
||||
- ? **Datenbank:** EF Core + SQLite (Tenant-Management)
|
||||
- ? **Background Services:** Job Processing, Temp-File Cleanup
|
||||
- ? Options Pattern Classes (Settings)
|
||||
- ? Options Pattern Classes (Settings)
|
||||
|
||||
**Was NICHT hierher gehört:**
|
||||
- ? Business Logic (? Application)
|
||||
@@ -1440,20 +1435,20 @@ Jedes Feature folgt dem gleichen Pattern:
|
||||
|
||||
---
|
||||
|
||||
### ? PHASE 6.5: Async Processing (Queue-based) - **NEW!**
|
||||
### ? PHASE 6.5: Async Processing (In-Memory Queue-based) - **NEW!**
|
||||
|
||||
**Ziel:** Große Operationen asynchron verarbeiten (> 5 Sekunden)
|
||||
|
||||
**Warum?**
|
||||
- ConcatenatePdfs von 50 PDFs = 10+ Sekunden
|
||||
- Client wartet nicht ? HTTP Timeout
|
||||
- Queue-basiert = skalierbar (Background Worker horizontal skalieren)
|
||||
- Queue-basiert = skalierbar (Background Worker kann parallel verarbeiten)
|
||||
|
||||
---
|
||||
|
||||
#### ? Step 6.5.1: Azure Storage Queue Setup
|
||||
#### ? Step 6.5.1: In-Memory Queue Setup
|
||||
|
||||
**Aufgabe:** Queue für Async Jobs
|
||||
**Aufgabe:** Queue für Async Jobs (In-Memory, keine Cloud-Abhängigkeiten)
|
||||
|
||||
**Was du erstellen wirst:**
|
||||
1. **IJobQueue Interface (Application):**
|
||||
@@ -1468,34 +1463,56 @@ Jedes Feature folgt dem gleichen Pattern:
|
||||
JobId JobId,
|
||||
ProcessingStatus Status,
|
||||
int Progress,
|
||||
string? ResultBlobPath,
|
||||
string? ResultFilePath,
|
||||
string? ErrorMessage);
|
||||
```
|
||||
|
||||
2. **AzureStorageJobQueue Implementation (Infrastructure):**
|
||||
2. **InMemoryJobQueue Implementation (Infrastructure):**
|
||||
```csharp
|
||||
public class AzureStorageJobQueue : IJobQueue
|
||||
public class InMemoryJobQueue : IJobQueue
|
||||
{
|
||||
private readonly QueueClient _queueClient;
|
||||
private readonly TableClient _tableClient; // Job Status Tracking
|
||||
private readonly ConcurrentQueue<JobData> _queue = new();
|
||||
private readonly ConcurrentDictionary<string, JobStatus> _jobStatuses = new();
|
||||
|
||||
public async Task<JobId> EnqueueAsync<T>(T jobData)
|
||||
public async Task<JobId> EnqueueAsync<T>(T jobData) where T : class
|
||||
{
|
||||
var jobId = JobId.Create(Guid.NewGuid().ToString());
|
||||
|
||||
// Job Message in Queue
|
||||
await _queueClient.SendMessageAsync(JsonSerializer.Serialize(jobData));
|
||||
|
||||
// Job Status in Table Storage (Pending)
|
||||
await _tableClient.AddEntityAsync(new JobStatusEntity
|
||||
_queue.Enqueue(new JobData
|
||||
{
|
||||
PartitionKey = jobId.Value,
|
||||
RowKey = jobId.Value,
|
||||
Status = ProcessingStatus.Pending
|
||||
JobId = jobId,
|
||||
Data = jobData,
|
||||
Type = typeof(T)
|
||||
});
|
||||
|
||||
// Job Status setzen (Pending)
|
||||
_jobStatuses[jobId.Value] = new JobStatus(
|
||||
jobId,
|
||||
ProcessingStatus.Pending,
|
||||
Progress: 0,
|
||||
ResultFilePath: null,
|
||||
ErrorMessage: null
|
||||
);
|
||||
|
||||
return jobId;
|
||||
}
|
||||
|
||||
public Task<JobStatus> GetStatusAsync(JobId jobId)
|
||||
{
|
||||
_jobStatuses.TryGetValue(jobId.Value, out var status);
|
||||
return Task.FromResult(status ?? throw new NotFoundException($"Job {jobId.Value} not found"));
|
||||
}
|
||||
|
||||
public bool TryDequeue(out JobData jobData)
|
||||
{
|
||||
return _queue.TryDequeue(out jobData);
|
||||
}
|
||||
|
||||
public void UpdateStatus(JobId jobId, JobStatus status)
|
||||
{
|
||||
_jobStatuses[jobId.Value] = status;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -1510,7 +1527,7 @@ Jedes Feature folgt dem gleichen Pattern:
|
||||
```csharp
|
||||
public class JobProcessorService : BackgroundService
|
||||
{
|
||||
private readonly QueueClient _queueClient;
|
||||
private readonly InMemoryJobQueue _jobQueue;
|
||||
private readonly IPdfProcessor _pdfProcessor;
|
||||
private readonly IFileStorage _fileStorage;
|
||||
|
||||
@@ -1519,35 +1536,48 @@ Jedes Feature folgt dem gleichen Pattern:
|
||||
while (!ct.IsCancellationRequested)
|
||||
{
|
||||
// Queue Message abrufen
|
||||
var message = await _queueClient.ReceiveMessageAsync();
|
||||
|
||||
if (message.Value != null)
|
||||
if (_jobQueue.TryDequeue(out var jobData))
|
||||
{
|
||||
// Job verarbeiten
|
||||
var jobData = JsonSerializer.Deserialize<ConcatenateJobData>(message.Value.Body);
|
||||
|
||||
try
|
||||
{
|
||||
// PDF-Operation
|
||||
var result = await _pdfProcessor.ConcatenateAsync(jobData.PdfFiles);
|
||||
// Job Status: Processing
|
||||
_jobQueue.UpdateStatus(jobData.JobId, new JobStatus(
|
||||
jobData.JobId,
|
||||
ProcessingStatus.Processing,
|
||||
Progress: 0,
|
||||
ResultFilePath: null,
|
||||
ErrorMessage: null
|
||||
));
|
||||
|
||||
// Ergebnis in Blob Storage
|
||||
var blobPath = await _fileStorage.SaveAsync(result, $"results/{jobData.JobId}.pdf");
|
||||
// PDF-Operation (z.B. Concatenate)
|
||||
var result = await _pdfProcessor.ConcatenateAsync(jobData.Data.PdfFiles);
|
||||
|
||||
// Ergebnis in lokalem Temp-Ordner speichern
|
||||
var resultPath = await _fileStorage.SaveAsync(result, $"results/{jobData.JobId}.pdf");
|
||||
|
||||
// Job Status: Success
|
||||
await UpdateJobStatusAsync(jobData.JobId, ProcessingStatus.Success, blobPath);
|
||||
|
||||
// Message löschen
|
||||
await _queueClient.DeleteMessageAsync(message.Value.MessageId, message.Value.PopReceipt);
|
||||
_jobQueue.UpdateStatus(jobData.JobId, new JobStatus(
|
||||
jobData.JobId,
|
||||
ProcessingStatus.Success,
|
||||
Progress: 100,
|
||||
ResultFilePath: resultPath,
|
||||
ErrorMessage: null
|
||||
));
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// Job Status: Failed
|
||||
await UpdateJobStatusAsync(jobData.JobId, ProcessingStatus.Failed, errorMessage: ex.Message);
|
||||
_jobQueue.UpdateStatus(jobData.JobId, new JobStatus(
|
||||
jobData.JobId,
|
||||
ProcessingStatus.Failed,
|
||||
Progress: 0,
|
||||
ResultFilePath: null,
|
||||
ErrorMessage: ex.Message
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
await Task.Delay(TimeSpan.FromSeconds(1), ct); // Polling-Interval
|
||||
await Task.Delay(TimeSpan.FromMilliseconds(100), ct); // Polling-Interval
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1606,13 +1636,13 @@ Client: GET /api/v1/jobs/abc123
|
||||
? API: { "jobId": "abc123", "status": "Processing", "progress": 50 }
|
||||
|
||||
Background Worker: Job fertig
|
||||
? Status-Update: Success (ResultBlobPath: "/results/abc123.pdf")
|
||||
? Status-Update: Success (ResultFilePath: "C:\\Temp\\DocumentOperator\\results\\abc123.pdf")
|
||||
|
||||
Client: GET /api/v1/jobs/abc123
|
||||
? API: { "jobId": "abc123", "status": "Success", "resultUrl": "/download/abc123" }
|
||||
|
||||
Client: GET /download/abc123
|
||||
? API: PDF-Datei aus Blob Storage
|
||||
? API: PDF-Datei aus lokalem Temp-Ordner
|
||||
```
|
||||
|
||||
---
|
||||
@@ -1848,11 +1878,16 @@ Client: GET /download/abc123
|
||||
|
||||
---
|
||||
|
||||
### ? PHASE 9: File Storage (Azure Blob) - **NEW!**
|
||||
### ? PHASE 9: File Storage (Lokale Temp-Ordner) - **NEW!**
|
||||
|
||||
**Ziel:** Multi-Server fähiges File Storage
|
||||
**Ziel:** File Storage für Temp-Files, Logos, Zertifikate
|
||||
|
||||
**Steps:**
|
||||
**Warum lokale Temp-Ordner?**
|
||||
- ? **Einfachheit:** Keine Cloud-Abhängigkeiten
|
||||
- ? **Schnell:** Lokaler Dateizugriff (keine Netzwerk-Latenz)
|
||||
- ? **Flexibel:** Bei Bedarf später zu Cloud migrierbar (IFileStorage bleibt!)
|
||||
|
||||
---
|
||||
|
||||
#### ? Step 9.1: IFileStorage Interface
|
||||
|
||||
@@ -1872,55 +1907,20 @@ Client: GET /download/abc123
|
||||
|
||||
---
|
||||
|
||||
#### ? Step 9.2: Azure Blob Storage Implementation
|
||||
#### ? Step 9.2: LocalFileStorage Implementation
|
||||
|
||||
**Aufgabe:** Azure Blob Storage Provider
|
||||
**Aufgabe:** Lokaler File Storage Provider
|
||||
|
||||
**Was du erstellen wirst:**
|
||||
1. **AzureBlobFileStorage.cs (Infrastructure/Services/FileStorage/):**
|
||||
```csharp
|
||||
public class AzureBlobFileStorage : IFileStorage
|
||||
{
|
||||
private readonly BlobContainerClient _containerClient;
|
||||
|
||||
public AzureBlobFileStorage(AzureBlobSettings settings)
|
||||
{
|
||||
var serviceClient = new BlobServiceClient(settings.ConnectionString);
|
||||
_containerClient = serviceClient.GetBlobContainerClient(settings.ContainerName);
|
||||
_containerClient.CreateIfNotExists();
|
||||
}
|
||||
|
||||
public async Task<string> SaveAsync(byte[] content, string filename)
|
||||
{
|
||||
var blobClient = _containerClient.GetBlobClient(filename);
|
||||
await blobClient.UploadAsync(new BinaryData(content), overwrite: true);
|
||||
return blobClient.Uri.ToString();
|
||||
}
|
||||
|
||||
public async Task<byte[]> GetAsync(string path)
|
||||
{
|
||||
var blobClient = _containerClient.GetBlobClient(path);
|
||||
var response = await blobClient.DownloadContentAsync();
|
||||
return response.Value.Content.ToArray();
|
||||
}
|
||||
|
||||
public async Task DeleteAsync(string path)
|
||||
{
|
||||
var blobClient = _containerClient.GetBlobClient(path);
|
||||
await blobClient.DeleteIfExistsAsync();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **LocalFileStorage.cs (für Dev/Test):**
|
||||
1. **LocalFileStorage.cs (Infrastructure/Services/FileStorage/):**
|
||||
```csharp
|
||||
public class LocalFileStorage : IFileStorage
|
||||
{
|
||||
private readonly string _basePath;
|
||||
|
||||
public LocalFileStorage()
|
||||
public LocalFileStorage(FileStorageSettings settings)
|
||||
{
|
||||
_basePath = Path.Combine(Directory.GetCurrentDirectory(), "LocalStorage");
|
||||
_basePath = settings.TempFolderPath ?? Path.Combine(Directory.GetCurrentDirectory(), "TempFiles");
|
||||
Directory.CreateDirectory(_basePath);
|
||||
}
|
||||
|
||||
@@ -1934,14 +1934,47 @@ Client: GET /download/abc123
|
||||
|
||||
public async Task<byte[]> GetAsync(string path)
|
||||
{
|
||||
return await File.ReadAllBytesAsync(path);
|
||||
// Pfad kann absolut oder relativ sein
|
||||
var fullPath = Path.IsPathFullyQualified(path) ? path : Path.Combine(_basePath, path);
|
||||
return await File.ReadAllBytesAsync(fullPath);
|
||||
}
|
||||
|
||||
public Task DeleteAsync(string path)
|
||||
{
|
||||
File.Delete(path);
|
||||
var fullPath = Path.IsPathFullyQualified(path) ? path : Path.Combine(_basePath, path);
|
||||
if (File.Exists(fullPath))
|
||||
{
|
||||
File.Delete(fullPath);
|
||||
}
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public Task<bool> ExistsAsync(string path)
|
||||
{
|
||||
var fullPath = Path.IsPathFullyQualified(path) ? path : Path.Combine(_basePath, path);
|
||||
return Task.FromResult(File.Exists(fullPath));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **FileStorageSettings.cs (Infrastructure/Configuration/):**
|
||||
```csharp
|
||||
public class FileStorageSettings
|
||||
{
|
||||
public string TempFolderPath { get; set; } = "TempFiles"; // Default: ./TempFiles
|
||||
public int CleanupIntervalHours { get; set; } = 24; // Default: täglich
|
||||
public int FileRetentionHours { get; set; } = 24; // Default: 24h Aufbewahrung
|
||||
}
|
||||
```
|
||||
|
||||
3. **appsettings.json:**
|
||||
```json
|
||||
{
|
||||
"FileStorage": {
|
||||
"TempFolderPath": "C:\\Temp\\DocumentOperator", // Windows Pfad
|
||||
"CleanupIntervalHours": 24,
|
||||
"FileRetentionHours": 24
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -1957,30 +1990,42 @@ Client: GET /download/abc123
|
||||
public class TempFileCleanupService : BackgroundService
|
||||
{
|
||||
private readonly IFileStorage _fileStorage;
|
||||
private readonly FileStorageSettings _settings;
|
||||
|
||||
protected override async Task ExecuteAsync(CancellationToken ct)
|
||||
{
|
||||
while (!ct.IsCancellationRequested)
|
||||
{
|
||||
// Täglich um 3 Uhr nachts
|
||||
var now = DateTime.UtcNow;
|
||||
var next3AM = now.Date.AddDays(1).AddHours(3);
|
||||
var delay = next3AM - now;
|
||||
// Warten auf nächsten Cleanup-Zeitpunkt
|
||||
await Task.Delay(TimeSpan.FromHours(_settings.CleanupIntervalHours), ct);
|
||||
|
||||
await Task.Delay(delay, ct);
|
||||
try
|
||||
{
|
||||
// Temp-Files älter als FileRetentionHours löschen
|
||||
var tempFolderPath = _settings.TempFolderPath;
|
||||
if (Directory.Exists(tempFolderPath))
|
||||
{
|
||||
var files = Directory.GetFiles(tempFolderPath, "*.*", SearchOption.AllDirectories);
|
||||
var cutoffTime = DateTime.UtcNow.AddHours(-_settings.FileRetentionHours);
|
||||
|
||||
// Temp-Files älter als 24h löschen
|
||||
var tempFiles = await _fileStorage.ListAsync("temp/");
|
||||
foreach (var file in tempFiles)
|
||||
foreach (var file in files)
|
||||
{
|
||||
if (file.CreatedAt < DateTime.UtcNow.AddHours(-24))
|
||||
var fileInfo = new FileInfo(file);
|
||||
if (fileInfo.CreationTimeUtc < cutoffTime)
|
||||
{
|
||||
await _fileStorage.DeleteAsync(file.Path);
|
||||
Log.Information("Deleted temp file: {Path}", file.Path);
|
||||
await _fileStorage.DeleteAsync(file);
|
||||
Log.Information("Deleted temp file: {Path} (Age: {Hours}h)",
|
||||
file, (DateTime.UtcNow - fileInfo.CreationTimeUtc).TotalHours);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Log.Error(ex, "Error during temp file cleanup");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -2060,15 +2105,16 @@ Client: GET /download/abc123
|
||||
.CreateLogger();
|
||||
```
|
||||
|
||||
2. **appsettings.Production.json (Application Insights):**
|
||||
2. **appsettings.Production.json (File Logging):**
|
||||
```json
|
||||
{
|
||||
"Serilog": {
|
||||
"WriteTo": [
|
||||
{
|
||||
"Name": "ApplicationInsights",
|
||||
"Name": "File",
|
||||
"Args": {
|
||||
"connectionString": "InstrumentationKey=..."
|
||||
"path": "C:\\Logs\\DocumentOperator\\log-.txt",
|
||||
"rollingInterval": "Day"
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -2083,13 +2129,13 @@ Client: GET /download/abc123
|
||||
**Ziel:** IIS Deployment + Production Configuration
|
||||
|
||||
**Steps:**
|
||||
- [ ] appsettings.Production.json (Azure Blob, Redis, Application Insights)
|
||||
- [ ] appsettings.Production.json (Lokale Temp-Ordner, Redis, File Logging)
|
||||
- [ ] IIS Web.config (Kestrel Settings)
|
||||
- [ ] SSL/TLS Configuration
|
||||
- [ ] Redis Connection String (Production)
|
||||
- [ ] Azure Blob Storage Connection String
|
||||
- [ ] Health Checks für Kubernetes/Load Balancer
|
||||
- [ ] Rate-Limiting (Redis-based, pro Tenant)
|
||||
- [ ] Redis Connection String (Production) - OPTIONAL (In-Memory Cache Alternative)
|
||||
- [ ] Shared Network Drive (bei Multi-Server Setup) - OPTIONAL (lokale Temp-Ordner für Single-Server)
|
||||
- [ ] Health Checks für Load Balancer
|
||||
- [ ] Rate-Limiting (Redis-based, pro Tenant) - OPTIONAL (In-Memory für Single-Server)
|
||||
|
||||
---
|
||||
|
||||
@@ -2242,8 +2288,8 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
|
||||
- EmbedCertificate
|
||||
|
||||
- **Phase 6.5:** Async Processing (**NEU!**)
|
||||
- Step 6.5.1 - Azure Storage Queue Setup
|
||||
- Step 6.5.2 - Background Worker (JobProcessorService)
|
||||
- Step 6.5.1 - In-Memory Queue Setup (IJobQueue Interface + InMemoryJobQueue)
|
||||
- Step 6.5.2 - Background Worker (JobProcessorService - IHostedService)
|
||||
- Step 6.5.3 - Async Endpoints (POST ? JobId, GET ? Status)
|
||||
|
||||
- **Phase 7:** Swagger & API Documentation (erweitert!)
|
||||
@@ -2257,12 +2303,12 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
|
||||
|
||||
- **Phase 9:** File Storage (**NEU!**)
|
||||
- Step 9.1 - IFileStorage Interface
|
||||
- Step 9.2 - Azure Blob Storage + Local File Storage
|
||||
- Step 9.3 - Temp-File Cleanup Service
|
||||
- Step 9.2 - LocalFileStorage Implementation (lokale Temp-Ordner)
|
||||
- Step 9.3 - Temp-File Cleanup Service (IHostedService)
|
||||
|
||||
- **Phase 10:** Logging & Monitoring (**NEU!**)
|
||||
- Step 10.1 - Correlation IDs (Request-Tracking)
|
||||
- Step 10.2 - Serilog Configuration (Seq + Application Insights)
|
||||
- Step 10.2 - Serilog Configuration (Seq + File Logging)
|
||||
|
||||
- **Phase 11:** Production Deployment
|
||||
- appsettings.Production.json
|
||||
@@ -2383,35 +2429,44 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
|
||||
|
||||
---
|
||||
|
||||
### 7. Async Processing: Queue-based (für große Operationen) - **NEU!**
|
||||
### 7. Async Processing: In-Memory Queue-based (für große Operationen) - **NEU!**
|
||||
|
||||
**Entscheidung:** Azure Storage Queue + Background Worker
|
||||
**Entscheidung:** In-Memory Queue + Background Worker
|
||||
|
||||
**Warum:**
|
||||
- ConcatenatePdfs von 50 PDFs = 10+ Sekunden
|
||||
- Synchron = HTTP Timeout
|
||||
- Queue = skalierbar (Worker horizontal skalieren)
|
||||
- In-Memory Queue = einfach, keine Cloud-Abhängigkeiten
|
||||
- ProcessingStatus Enum wird jetzt genutzt!
|
||||
|
||||
**Alternative wäre gewesen:**
|
||||
- Alle Operationen synchron
|
||||
- **Nachteile:** HTTP Timeouts, nicht skalierbar
|
||||
|
||||
**Hinweis:**
|
||||
- In-Memory Queue = nicht persistent (bei Server-Neustart gehen Jobs verloren)
|
||||
- Für Production später: Redis Queue oder RabbitMQ (aber IJobQueue bleibt!)
|
||||
- Für Single-Server Setup: In-Memory Queue ausreichend
|
||||
|
||||
---
|
||||
|
||||
### 8. File Storage: Azure Blob (statt Local Files) - **NEU!**
|
||||
### 8. File Storage: Lokale Temp-Ordner (statt Cloud) - **NEU!**
|
||||
|
||||
**Entscheidung:** IFileStorage Interface + Azure Blob + Local (Dev)
|
||||
**Entscheidung:** IFileStorage Interface + LocalFileStorage
|
||||
|
||||
**Warum:**
|
||||
- **Multi-Server:** Load Balancer mit 3 API-Instanzen
|
||||
- **Shared Storage:** Azure Blob = alle Server greifen auf gleiche Files zu
|
||||
- **Abstraction:** LocalFileStorage für Dev/Test
|
||||
- **Automatic Cleanup:** TempFileCleanupService (IHostedService)
|
||||
- **Einfachheit:** Keine Cloud-Abhängigkeiten (Azure Blob Storage)
|
||||
- **Schnell:** Lokaler Dateizugriff (keine Netzwerk-Latenz)
|
||||
- **Flexibel:** IFileStorage Abstraction bleibt (später zu Cloud migrierbar!)
|
||||
- **TempFileCleanupService:** Automatische Cleanup (täglich)
|
||||
|
||||
**Alternative wäre gewesen:**
|
||||
- Temp-Files auf lokalem Server
|
||||
- **Nachteile:** Multi-Server nicht möglich, Disk voll nach 1 Monat
|
||||
- Cloud Storage (Azure Blob, AWS S3)
|
||||
- **Nachteile:** Cloud-Abhängigkeit, Kosten, Komplexität
|
||||
|
||||
**Hinweis:**
|
||||
- Für Multi-Server Setup: Shared Network Drive (UNC-Pfad) statt lokale Ordner
|
||||
- IFileStorage Interface bleibt gleich (austauschbar!)
|
||||
|
||||
---
|
||||
|
||||
@@ -2445,19 +2500,19 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
|
||||
|
||||
---
|
||||
|
||||
### 11. Logging: Correlation IDs + Seq + Application Insights - **NEU!**
|
||||
### 11. Logging: Correlation IDs + Seq + File Logging - **NEU!**
|
||||
|
||||
**Entscheidung:** Structured Logging mit Correlation IDs
|
||||
|
||||
**Warum:**
|
||||
- **Correlation IDs:** Request-Tracking über alle Logs (Debugging leichter)
|
||||
- **Seq:** Log-Browsing UI (Development)
|
||||
- **Application Insights:** Production Monitoring (Azure)
|
||||
- **File Logging:** Production Logs (keine Cloud-Abhängigkeit)
|
||||
- **LoggingBehavior:** MediatR Pipeline Behavior (automatisches Logging)
|
||||
|
||||
**Alternative wäre gewesen:**
|
||||
- Nur File Logging (keine Correlation IDs)
|
||||
- **Nachteile:** Debugging schwierig, keine Request-Zusammenhänge
|
||||
- Nur Console Logging (keine Correlation IDs)
|
||||
- **Nachteile:** Debugging schwierig, keine Request-Zusammenhänge, keine persistente Log-Speicherung
|
||||
|
||||
### 5. Vertical Slice Architecture
|
||||
|
||||
@@ -2488,8 +2543,6 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
|
||||
- [FluentAssertions Documentation](https://fluentassertions.com/)
|
||||
- [Polly Documentation](https://www.pollydocs.org/) - **NEU:** Resilience Patterns
|
||||
- [EF Core Documentation](https://learn.microsoft.com/en-us/ef/core/) - **NEU:** ORM für Tenant-DB
|
||||
- [Azure Blob Storage Documentation](https://learn.microsoft.com/en-us/azure/storage/blobs/) - **NEU:** File Storage
|
||||
- [Azure Storage Queue Documentation](https://learn.microsoft.com/en-us/azure/storage/queues/) - **NEU:** Async Processing
|
||||
- [Serilog Documentation](https://serilog.net/) - **NEU:** Structured Logging
|
||||
- [Seq Documentation](https://docs.datalust.co/docs) - **NEU:** Log Browsing UI
|
||||
|
||||
@@ -2504,17 +2557,17 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
|
||||
- ? TDD (Test-Driven Development)
|
||||
- ? **Resilience Patterns (Polly)** - **NEU:** Retry, Circuit Breaker, Timeout
|
||||
- ? **Multi-Tenancy (Database-based)** - **NEU:** EF Core + SQLite + Redis Cache
|
||||
- ? **Async Processing (Queue-based)** - **NEU:** Azure Storage Queue + Background Worker
|
||||
- ? **File Storage Abstraction (Azure Blob)** - **NEU:** Multi-Server fähig
|
||||
- ? **Async Processing (In-Memory Queue)** - **NEU:** In-Memory Queue + Background Worker
|
||||
- ? **File Storage Abstraction (Lokale Temp-Ordner)** - **NEU:** IFileStorage + LocalFileStorage
|
||||
- ? **Correlation IDs** - **NEU:** Request-Tracking über alle Logs
|
||||
- ? **Health Checks** - **NEU:** Kubernetes/Load Balancer Support
|
||||
- ? **Health Checks** - **NEU:** Load Balancer Support
|
||||
- ? Options Pattern für Configuration
|
||||
- ? Dependency Injection
|
||||
- ? Async/Await überall
|
||||
- ? Nullable Reference Types
|
||||
- ? Record Types für DTOs (C# 12)
|
||||
- ? Primary Constructors (.NET 8)
|
||||
- ? Structured Logging (Serilog + Seq + Application Insights)
|
||||
- ? Structured Logging (Serilog + Seq + File Logging)
|
||||
|
||||
---
|
||||
|
||||
@@ -2540,14 +2593,16 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
|
||||
| 17.01.2025 | Tests | ? Step 3.2.4 - DevExpressPdfProcessorTests.cs erstellt (TDD Red - 6 Tests) |
|
||||
| 17.01.2025 | Roadmap | ?? **ROADMAP MAJOR UPDATE** - Production-Ready Features hinzugefügt! |
|
||||
| 17.01.2025 | Architecture | ? **Multi-Tenancy:** Database-based (EF Core + SQLite + Redis Cache) |
|
||||
| 17.01.2025 | Architecture | ? **Async Processing:** Queue-based (Azure Storage Queue + Background Worker) |
|
||||
| 17.01.2025 | Architecture | ? **File Storage:** Azure Blob Storage + IFileStorage Abstraction |
|
||||
| 17.01.2025 | Architecture | ? **Async Processing:** In-Memory Queue-based + Background Worker |
|
||||
| 17.01.2025 | Architecture | ? **File Storage:** Lokale Temp-Ordner mit IFileStorage Abstraction |
|
||||
| 17.01.2025 | Architecture | ? **Resilience:** Polly (Retry, Circuit Breaker, Timeout) |
|
||||
| 17.01.2025 | Architecture | ? **Health Checks:** Früh implementieren (Phase 5.5 statt Phase 9) |
|
||||
| 17.01.2025 | Architecture | ? **Logging:** Correlation IDs + Seq + Application Insights |
|
||||
| 17.01.2025 | Technology Stack | ? **11 neue NuGet Packages hinzugefügt** (EF Core, Polly, Azure Storage, BCrypt, Seq) |
|
||||
| 17.01.2025 | Architecture | ? **Logging:** Correlation IDs + Seq + File Logging |
|
||||
| 17.01.2025 | Technology Stack | ? **9 neue NuGet Packages hinzugefügt** (EF Core, Polly, BCrypt, Seq - OHNE Azure) |
|
||||
| 17.01.2025 | Roadmap | ? **6 neue Phasen** (5.5, 6.5, 8, 9, 10, 11) - insgesamt 11 Phasen statt 9 |
|
||||
| 17.01.2025 | Documentation | ? **11 Key Learnings & Decisions** dokumentiert (statt 5)
|
||||
| 17.01.2025 | Documentation | ? **10 Key Learnings & Decisions** dokumentiert (statt 5) |
|
||||
| 17.01.2025 | Roadmap | ?? **Azure Services entfernt** - Lokale Temp-Ordner + In-Memory Queue stattdessen
|
||||
| 22.06.2026 | Dokumentation | ? **Azure-Referenzen vollständig entfernt** - Alle Azure-Referenzen bereinigt (PROJECT_STATUS.md + ROADMAP.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user