AppStd/DocumentService
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor: Remove Azure dependencies, use local storage

Browse Source 
Replaced Azure Blob Storage and Storage Queue with local
temp folders and an in-memory queue for file storage and
async processing. Updated `IFileStorage` and `IJobQueue`
interfaces to support the new architecture.

Modified `TenantSettings` and `ApplyStampHandler` to use
local file paths. Updated `JobProcessorService` to handle
in-memory queue jobs. Added file cleanup policies to
`LocalFileStorage`.

Revised roadmap and documentation to reflect the shift
to local-first architecture, emphasizing simplicity,
reduced cloud dependencies, and single-server readiness.
Logging now uses file-based storage instead of Application
Insights. Adjusted production deployment and health check
phases to align with the new approach.
This commit is contained in:
OlgunR
2026-06-22 14:14:57 +02:00
parent d50e30f7ac
commit 10cfb0c838
2 changed files with 616 additions and 162 deletions
Download Patch File Download Diff File Expand all files Collapse all files

379
DocumentOperator.API/ROADMAP.md
View File

@@ -1,6 +1,6 @@
# ?? DocumentOperator - Project Roadmap (Pragmatic Edition)
> **Last Updated:** 17.01.2025 (Updated with Resilience, Async Processing, Multi-Tenancy DB) | **Status:** In Development | **Phase:** 3 (Infrastructure Layer)
> **Last Updated:** 22.06.2026 (Azure-Referenzen vollständig entfernt) | **Status:** In Development | **Phase:** 3 (Infrastructure Layer)
---
@@ -9,22 +9,22 @@
**Was ist neu in diesem Update?**
1. **? Multi-Tenancy:** Database-based (EF Core + SQLite + Redis Cache) statt appsettings.json
2. **? Async Processing:** Queue-based (Azure Storage Queue + Background Worker) für große Operationen
3. **? File Storage:** Azure Blob Storage + IFileStorage Abstraction (Multi-Server fähig!)
2. **? Async Processing:** In-Memory Queue-based + Background Worker für große Operationen
3. **? File Storage:** Lokale Temp-Ordner mit IFileStorage Abstraction
4. **? Resilience:** Polly (Retry, Circuit Breaker, Timeout) für DevExpress Calls
5. **?? Health Checks:** FRÜH implementieren (Phase 5.5 statt Phase 9)
6. **? Logging:** Correlation IDs + Seq + Application Insights
7. **? 11 neue NuGet Packages:** EF Core, Polly, Azure Storage, BCrypt, Seq, Correlation IDs
6. **? Logging:** Correlation IDs + Seq + File Logging
7. **? 9 neue NuGet Packages:** EF Core, Polly, BCrypt, Seq, Correlation IDs
8. **? 6 neue Phasen:** 5.5, 6.5, 8, 9, 10, 11 (insgesamt 11 Phasen statt 9)
9. **? 11 Key Learnings:** Dokumentiert (statt 5)
10. **? Technology Stack:** Komplett aktualisiert mit allen neuen Dependencies
**Warum diese Änderungen?**
- **Skalierbarkeit:** Multi-Server Support (Azure Blob, Redis Cache)
- **Einfachheit:** Lokale Temp-Ordner (keine Cloud-Abhängigkeiten)
- **Security:** API-Key Hashing (BCrypt), Rotation möglich
- **Performance:** Async Processing für große Operationen (keine HTTP Timeouts)
- **Resilience:** Production-ready (Polly Retry/Circuit Breaker)
- **Monitoring:** Correlation IDs, Seq, Application Insights
- **Monitoring:** Correlation IDs, Seq, File Logging
- **Wartbarkeit:** Clean Architecture bleibt pragmatisch, aber production-ready!
---
@@ -120,7 +120,7 @@ Client Application
?
[Returns: { "jobId": "abc123", "status": "Pending" }] - Sofort
?
Background Worker (IHostedService) ? Azure Storage Queue ? Verarbeitung
Background Worker (IHostedService) ? In-Memory Queue ? Verarbeitung
?
Client Poll: [GET /api/v1/jobs/abc123] ? { "status": "Processing", "progress": 45% }
?
@@ -133,7 +133,7 @@ Client Poll: [GET /api/v1/jobs/abc123] ? { "status": "Success", "resultUrl": "/d
3. API validiert Input (FluentValidation in MediatR Pipeline)
4. Handler konvertiert PDF ? Byte-Array
5. DevExpress Service führt Operation durch (mit Polly Retry/Circuit Breaker)
6. Ergebnis wird in Azure Blob Storage gespeichert (Multi-Server!)
6. Ergebnis wird in lokalem Temp-Ordner gespeichert
7. Ergebnis wird als Base64 zurückgegeben (oder Download-Link)
---
@@ -455,8 +455,8 @@ public class TenantSettings
{
public Guid Id { get; set; }
public Guid TenantId { get; set; }
public string LogoBlobPath { get; set; } // Azure Blob Storage Pfad
public string CertificateBlobPath { get; set; } // Azure Blob Storage Pfad
public string LogoFilePath { get; set; } // Lokaler Dateipfad (z.B. "logos/tenant-a/stamp.png")
public string CertificateFilePath { get; set; } // Lokaler Dateipfad (z.B. "certs/tenant-a/cert.pfx")
public int RateLimitPerMinute { get; set; } // Rate-Limiting
// Navigation
@@ -469,14 +469,14 @@ public class TenantSettings
public class ApplyStampHandler : IRequestHandler<ApplyStampCommand, byte[]>
{
private readonly ITenantContext _tenantContext;
private readonly IFileStorage _fileStorage; // Azure Blob Storage
private readonly IFileStorage _fileStorage; // Lokaler File Storage
public async Task<byte[]> Handle(ApplyStampCommand command, CancellationToken ct)
{
// Tenant-spezifisches Logo aus DB Settings laden
var logoPath = _tenantContext.CurrentTenant.Settings.LogoBlobPath;
var logoPath = _tenantContext.CurrentTenant.Settings.LogoFilePath;
// Logo aus Azure Blob Storage laden
// Logo aus lokalem Dateisystem laden
var logoBytes = await _fileStorage.GetAsync(logoPath);
// Stamp mit Logo anwenden
@@ -537,7 +537,6 @@ public class ApplyStampHandler : IRequestHandler<ApplyStampCommand, byte[]>
| **Microsoft.EntityFrameworkCore** | 8.0.0 | **NEU:** ORM für Tenant-Datenbank |
| **Microsoft.EntityFrameworkCore.Sqlite** | 8.0.0 | **NEU:** SQLite Provider (Tenant-DB) |
| **Microsoft.EntityFrameworkCore.Tools** | 8.0.0 | **NEU:** Migrations-Support |
| **Azure.Storage.Blobs** | 12.22.3 | **NEU:** Azure Blob Storage (File Storage) |
| **Polly** | 8.5.0 | **NEU:** Resilience (Retry, Circuit Breaker, Timeout) |
| **BCrypt.Net-Next** | 4.0.3 | **NEU:** API-Key Hashing (Security) |
| **Microsoft.Extensions.Options.ConfigurationExtensions** | 8.0.0 | Options Pattern |
@@ -552,10 +551,9 @@ public class ApplyStampHandler : IRequestHandler<ApplyStampCommand, byte[]>
**Warum diese neuen Pakete?**
- **EF Core + SQLite:** Tenant-Datenbank (API-Keys, Settings) - skalierbar ohne SQL Server
- **Azure Blob Storage:** Multi-Server File Storage (Logos, Zertifikate, Temp-Files)
- **Polly:** Resilience für DevExpress Calls (Retry bei Transient Errors, Circuit Breaker)
- **BCrypt:** Sichere API-Key Hashes (NICHT Klartext in DB!)
- **Seq:** Log-Browsing UI für Development (Production: Application Insights)
- **Seq:** Log-Browsing UI für Development
- **Correlation IDs:** Request-Tracking über alle Logs (Debugging leichter)
#### Domain Layer
@@ -664,7 +662,7 @@ DocumentOperator.Application/
? ??? Interfaces/ ? Abstractions für Infrastructure
? ? ??? IPdfProcessor.cs
? ? ??? IFileStorage.cs ? **NEU:** File Storage Abstraction
? ? ??? IJobQueue.cs ? **NEU:** Queue Abstraction (Azure Storage Queue)
? ? ??? IJobQueue.cs ? **NEU:** Queue Abstraction (In-Memory Queue)
? ? ??? ITenantRepository.cs ? **NEU:** Tenant-DB Abstraction
? ??? Behaviors/ ? MediatR Pipeline Behaviors
? ? ??? ValidationBehavior.cs ? FluentValidation Integration
@@ -715,10 +713,9 @@ DocumentOperator.Infrastructure/
? ??? PdfProcessing/
? ? ??? DevExpressPdfProcessor.cs ? IPdfProcessor Implementation (mit Polly Resilience)
? ??? FileStorage/
? ? ??? AzureBlobFileStorage.cs ? **NEU:** IFileStorage Implementation (Azure Blob)
? ? ??? LocalFileStorage.cs ? **NEU:** IFileStorage Implementation (Dev/Test)
? ? ??? LocalFileStorage.cs ? **NEU:** IFileStorage Implementation (lokaler Temp-Ordner)
? ??? Queue/
? ??? AzureStorageJobQueue.cs ? **NEU:** IJobQueue Implementation (Azure Storage Queue)
? ??? InMemoryJobQueue.cs ? **NEU:** IJobQueue Implementation (In-Memory Queue)
??? Data/
? ??? TenantDbContext.cs ? **NEU:** EF Core DbContext (Tenant-DB)
? ??? Entities/
@@ -732,20 +729,18 @@ DocumentOperator.Infrastructure/
? ??? TempFileCleanupService.cs ? **NEU:** IHostedService für Temp-File Cleanup (täglich)
??? Configuration/
? ??? DocumentOperatorSettings.cs ? Options Pattern Class
? ??? AzureBlobSettings.cs ? **NEU:** Azure Blob Storage Configuration
? ??? AzureQueueSettings.cs ? **NEU:** Azure Storage Queue Configuration
? ??? FileStorageSettings.cs ? **NEU:** File Storage Configuration (Temp-Ordner Pfad)
? ??? RedisSettings.cs ? **NEU:** Redis Cache Configuration
??? DependencyInjection.cs ? Service Registration
```
**Was gehört hierher:**
- ? DevExpress Integration (mit Polly Resilience!)
- ? **File Storage:** Azure Blob Storage + Local File Storage (Abstraction!)
- ? **Queue:** Azure Storage Queue für Async Processing
- ? **File Storage:** Lokaler Temp-Ordner (IFileStorage Abstraction)
- ? **Queue:** In-Memory Queue für Async Processing
- ? **Datenbank:** EF Core + SQLite (Tenant-Management)
- ? **Background Services:** Job Processing, Temp-File Cleanup
- ? Options Pattern Classes (Settings)
- ? Options Pattern Classes (Settings)
**Was NICHT hierher gehört:**
- ? Business Logic (? Application)
@@ -1440,20 +1435,20 @@ Jedes Feature folgt dem gleichen Pattern:
---
### ? PHASE 6.5: Async Processing (Queue-based) - **NEW!**
### ? PHASE 6.5: Async Processing (In-Memory Queue-based) - **NEW!**
**Ziel:** Große Operationen asynchron verarbeiten (> 5 Sekunden)
**Warum?**
- ConcatenatePdfs von 50 PDFs = 10+ Sekunden
- Client wartet nicht ? HTTP Timeout
- Queue-basiert = skalierbar (Background Worker horizontal skalieren)
- Queue-basiert = skalierbar (Background Worker kann parallel verarbeiten)
---
#### ? Step 6.5.1: Azure Storage Queue Setup
#### ? Step 6.5.1: In-Memory Queue Setup
**Aufgabe:** Queue für Async Jobs
**Aufgabe:** Queue für Async Jobs (In-Memory, keine Cloud-Abhängigkeiten)
**Was du erstellen wirst:**
1. **IJobQueue Interface (Application):**
@@ -1468,34 +1463,56 @@ Jedes Feature folgt dem gleichen Pattern:
JobId JobId,
ProcessingStatus Status,
int Progress,
string? ResultBlobPath,
string? ResultFilePath,
string? ErrorMessage);
```
2. **AzureStorageJobQueue Implementation (Infrastructure):**
2. **InMemoryJobQueue Implementation (Infrastructure):**
```csharp
public class AzureStorageJobQueue : IJobQueue
public class InMemoryJobQueue : IJobQueue
{
private readonly QueueClient _queueClient;
private readonly TableClient _tableClient; // Job Status Tracking
private readonly ConcurrentQueue<JobData> _queue = new();
private readonly ConcurrentDictionary<string, JobStatus> _jobStatuses = new();
public async Task<JobId> EnqueueAsync<T>(T jobData)
public async Task<JobId> EnqueueAsync<T>(T jobData) where T : class
{
var jobId = JobId.Create(Guid.NewGuid().ToString());
// Job Message in Queue
await _queueClient.SendMessageAsync(JsonSerializer.Serialize(jobData));
// Job Status in Table Storage (Pending)
await _tableClient.AddEntityAsync(new JobStatusEntity
_queue.Enqueue(new JobData
{
PartitionKey = jobId.Value,
RowKey = jobId.Value,
Status = ProcessingStatus.Pending
JobId = jobId,
Data = jobData,
Type = typeof(T)
});
// Job Status setzen (Pending)
_jobStatuses[jobId.Value] = new JobStatus(
jobId,
ProcessingStatus.Pending,
Progress: 0,
ResultFilePath: null,
ErrorMessage: null
);
return jobId;
}
public Task<JobStatus> GetStatusAsync(JobId jobId)
{
_jobStatuses.TryGetValue(jobId.Value, out var status);
return Task.FromResult(status ?? throw new NotFoundException($"Job {jobId.Value} not found"));
}
public bool TryDequeue(out JobData jobData)
{
return _queue.TryDequeue(out jobData);
}
public void UpdateStatus(JobId jobId, JobStatus status)
{
_jobStatuses[jobId.Value] = status;
}
}
```
@@ -1510,7 +1527,7 @@ Jedes Feature folgt dem gleichen Pattern:
```csharp
public class JobProcessorService : BackgroundService
{
private readonly QueueClient _queueClient;
private readonly InMemoryJobQueue _jobQueue;
private readonly IPdfProcessor _pdfProcessor;
private readonly IFileStorage _fileStorage;
@@ -1519,38 +1536,51 @@ Jedes Feature folgt dem gleichen Pattern:
while (!ct.IsCancellationRequested)
{
// Queue Message abrufen
var message = await _queueClient.ReceiveMessageAsync();
if (message.Value != null)
if (_jobQueue.TryDequeue(out var jobData))
{
// Job verarbeiten
var jobData = JsonSerializer.Deserialize<ConcatenateJobData>(message.Value.Body);
try
{
// PDF-Operation
var result = await _pdfProcessor.ConcatenateAsync(jobData.PdfFiles);
// Job Status: Processing
_jobQueue.UpdateStatus(jobData.JobId, new JobStatus(
jobData.JobId,
ProcessingStatus.Processing,
Progress: 0,
ResultFilePath: null,
ErrorMessage: null
));
// Ergebnis in Blob Storage
var blobPath = await _fileStorage.SaveAsync(result, $"results/{jobData.JobId}.pdf");
// PDF-Operation (z.B. Concatenate)
var result = await _pdfProcessor.ConcatenateAsync(jobData.Data.PdfFiles);
// Ergebnis in lokalem Temp-Ordner speichern
var resultPath = await _fileStorage.SaveAsync(result, $"results/{jobData.JobId}.pdf");
// Job Status: Success
await UpdateJobStatusAsync(jobData.JobId, ProcessingStatus.Success, blobPath);
// Message löschen
await _queueClient.DeleteMessageAsync(message.Value.MessageId, message.Value.PopReceipt);
_jobQueue.UpdateStatus(jobData.JobId, new JobStatus(
jobData.JobId,
ProcessingStatus.Success,
Progress: 100,
ResultFilePath: resultPath,
ErrorMessage: null
));
}
catch (Exception ex)
{
// Job Status: Failed
await UpdateJobStatusAsync(jobData.JobId, ProcessingStatus.Failed, errorMessage: ex.Message);
_jobQueue.UpdateStatus(jobData.JobId, new JobStatus(
jobData.JobId,
ProcessingStatus.Failed,
Progress: 0,
ResultFilePath: null,
ErrorMessage: ex.Message
));
}
}
await Task.Delay(TimeSpan.FromSeconds(1), ct); // Polling-Interval
}
}
}
await Task.Delay(TimeSpan.FromMilliseconds(100), ct); // Polling-Interval
}
}
}
```
2. **Program.cs Registration:**
@@ -1606,13 +1636,13 @@ Client: GET /api/v1/jobs/abc123
? API: { "jobId": "abc123", "status": "Processing", "progress": 50 }
Background Worker: Job fertig
? Status-Update: Success (ResultBlobPath: "/results/abc123.pdf")
? Status-Update: Success (ResultFilePath: "C:\\Temp\\DocumentOperator\\results\\abc123.pdf")
Client: GET /api/v1/jobs/abc123
? API: { "jobId": "abc123", "status": "Success", "resultUrl": "/download/abc123" }
Client: GET /download/abc123
? API: PDF-Datei aus Blob Storage
? API: PDF-Datei aus lokalem Temp-Ordner
```
---
@@ -1848,11 +1878,16 @@ Client: GET /download/abc123
---
### ? PHASE 9: File Storage (Azure Blob) - **NEW!**
### ? PHASE 9: File Storage (Lokale Temp-Ordner) - **NEW!**
**Ziel:** Multi-Server fähiges File Storage
**Ziel:** File Storage für Temp-Files, Logos, Zertifikate
**Steps:**
**Warum lokale Temp-Ordner?**
- ? **Einfachheit:** Keine Cloud-Abhängigkeiten
- ? **Schnell:** Lokaler Dateizugriff (keine Netzwerk-Latenz)
- ? **Flexibel:** Bei Bedarf später zu Cloud migrierbar (IFileStorage bleibt!)
---
#### ? Step 9.1: IFileStorage Interface
@@ -1872,55 +1907,20 @@ Client: GET /download/abc123
---
#### ? Step 9.2: Azure Blob Storage Implementation
#### ? Step 9.2: LocalFileStorage Implementation
**Aufgabe:** Azure Blob Storage Provider
**Aufgabe:** Lokaler File Storage Provider
**Was du erstellen wirst:**
1. **AzureBlobFileStorage.cs (Infrastructure/Services/FileStorage/):**
```csharp
public class AzureBlobFileStorage : IFileStorage
{
private readonly BlobContainerClient _containerClient;
public AzureBlobFileStorage(AzureBlobSettings settings)
{
var serviceClient = new BlobServiceClient(settings.ConnectionString);
_containerClient = serviceClient.GetBlobContainerClient(settings.ContainerName);
_containerClient.CreateIfNotExists();
}
public async Task<string> SaveAsync(byte[] content, string filename)
{
var blobClient = _containerClient.GetBlobClient(filename);
await blobClient.UploadAsync(new BinaryData(content), overwrite: true);
return blobClient.Uri.ToString();
}
public async Task<byte[]> GetAsync(string path)
{
var blobClient = _containerClient.GetBlobClient(path);
var response = await blobClient.DownloadContentAsync();
return response.Value.Content.ToArray();
}
public async Task DeleteAsync(string path)
{
var blobClient = _containerClient.GetBlobClient(path);
await blobClient.DeleteIfExistsAsync();
}
}
```
2. **LocalFileStorage.cs (für Dev/Test):**
1. **LocalFileStorage.cs (Infrastructure/Services/FileStorage/):**
```csharp
public class LocalFileStorage : IFileStorage
{
private readonly string _basePath;
public LocalFileStorage()
public LocalFileStorage(FileStorageSettings settings)
{
_basePath = Path.Combine(Directory.GetCurrentDirectory(), "LocalStorage");
_basePath = settings.TempFolderPath ?? Path.Combine(Directory.GetCurrentDirectory(), "TempFiles");
Directory.CreateDirectory(_basePath);
}
@@ -1934,18 +1934,51 @@ Client: GET /download/abc123
public async Task<byte[]> GetAsync(string path)
{
return await File.ReadAllBytesAsync(path);
// Pfad kann absolut oder relativ sein
var fullPath = Path.IsPathFullyQualified(path) ? path : Path.Combine(_basePath, path);
return await File.ReadAllBytesAsync(fullPath);
}
public Task DeleteAsync(string path)
{
File.Delete(path);
var fullPath = Path.IsPathFullyQualified(path) ? path : Path.Combine(_basePath, path);
if (File.Exists(fullPath))
{
File.Delete(fullPath);
}
return Task.CompletedTask;
}
public Task<bool> ExistsAsync(string path)
{
var fullPath = Path.IsPathFullyQualified(path) ? path : Path.Combine(_basePath, path);
return Task.FromResult(File.Exists(fullPath));
}
}
```
---
2. **FileStorageSettings.cs (Infrastructure/Configuration/):**
```csharp
public class FileStorageSettings
{
public string TempFolderPath { get; set; } = "TempFiles"; // Default: ./TempFiles
public int CleanupIntervalHours { get; set; } = 24; // Default: täglich
public int FileRetentionHours { get; set; } = 24; // Default: 24h Aufbewahrung
}
```
3. **appsettings.json:**
```json
{
"FileStorage": {
"TempFolderPath": "C:\\Temp\\DocumentOperator", // Windows Pfad
"CleanupIntervalHours": 24,
"FileRetentionHours": 24
}
}
```
---
#### ? Step 9.3: Temp-File Cleanup Service
@@ -1957,28 +1990,40 @@ Client: GET /download/abc123
public class TempFileCleanupService : BackgroundService
{
private readonly IFileStorage _fileStorage;
private readonly FileStorageSettings _settings;
protected override async Task ExecuteAsync(CancellationToken ct)
{
while (!ct.IsCancellationRequested)
{
// Täglich um 3 Uhr nachts
var now = DateTime.UtcNow;
var next3AM = now.Date.AddDays(1).AddHours(3);
var delay = next3AM - now;
// Warten auf nächsten Cleanup-Zeitpunkt
await Task.Delay(TimeSpan.FromHours(_settings.CleanupIntervalHours), ct);
await Task.Delay(delay, ct);
// Temp-Files älter als 24h löschen
var tempFiles = await _fileStorage.ListAsync("temp/");
foreach (var file in tempFiles)
try
{
if (file.CreatedAt < DateTime.UtcNow.AddHours(-24))
// Temp-Files älter als FileRetentionHours löschen
var tempFolderPath = _settings.TempFolderPath;
if (Directory.Exists(tempFolderPath))
{
await _fileStorage.DeleteAsync(file.Path);
Log.Information("Deleted temp file: {Path}", file.Path);
var files = Directory.GetFiles(tempFolderPath, "*.*", SearchOption.AllDirectories);
var cutoffTime = DateTime.UtcNow.AddHours(-_settings.FileRetentionHours);
foreach (var file in files)
{
var fileInfo = new FileInfo(file);
if (fileInfo.CreationTimeUtc < cutoffTime)
{
await _fileStorage.DeleteAsync(file);
Log.Information("Deleted temp file: {Path} (Age: {Hours}h)",
file, (DateTime.UtcNow - fileInfo.CreationTimeUtc).TotalHours);
}
}
}
}
catch (Exception ex)
{
Log.Error(ex, "Error during temp file cleanup");
}
}
}
}
@@ -2060,15 +2105,16 @@ Client: GET /download/abc123
.CreateLogger();
```
2. **appsettings.Production.json (Application Insights):**
2. **appsettings.Production.json (File Logging):**
```json
{
"Serilog": {
"WriteTo": [
{
"Name": "ApplicationInsights",
"Name": "File",
"Args": {
"connectionString": "InstrumentationKey=..."
"path": "C:\\Logs\\DocumentOperator\\log-.txt",
"rollingInterval": "Day"
}
}
]
@@ -2083,13 +2129,13 @@ Client: GET /download/abc123
**Ziel:** IIS Deployment + Production Configuration
**Steps:**
- [ ] appsettings.Production.json (Azure Blob, Redis, Application Insights)
- [ ] appsettings.Production.json (Lokale Temp-Ordner, Redis, File Logging)
- [ ] IIS Web.config (Kestrel Settings)
- [ ] SSL/TLS Configuration
- [ ] Redis Connection String (Production)
- [ ] Azure Blob Storage Connection String
- [ ] Health Checks für Kubernetes/Load Balancer
- [ ] Rate-Limiting (Redis-based, pro Tenant)
- [ ] Redis Connection String (Production) - OPTIONAL (In-Memory Cache Alternative)
- [ ] Shared Network Drive (bei Multi-Server Setup) - OPTIONAL (lokale Temp-Ordner für Single-Server)
- [ ] Health Checks für Load Balancer
- [ ] Rate-Limiting (Redis-based, pro Tenant) - OPTIONAL (In-Memory für Single-Server)
---
@@ -2242,8 +2288,8 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
- EmbedCertificate
- **Phase 6.5:** Async Processing (**NEU!**)
- Step 6.5.1 - Azure Storage Queue Setup
- Step 6.5.2 - Background Worker (JobProcessorService)
- Step 6.5.1 - In-Memory Queue Setup (IJobQueue Interface + InMemoryJobQueue)
- Step 6.5.2 - Background Worker (JobProcessorService - IHostedService)
- Step 6.5.3 - Async Endpoints (POST ? JobId, GET ? Status)
- **Phase 7:** Swagger & API Documentation (erweitert!)
@@ -2257,12 +2303,12 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
- **Phase 9:** File Storage (**NEU!**)
- Step 9.1 - IFileStorage Interface
- Step 9.2 - Azure Blob Storage + Local File Storage
- Step 9.3 - Temp-File Cleanup Service
- Step 9.2 - LocalFileStorage Implementation (lokale Temp-Ordner)
- Step 9.3 - Temp-File Cleanup Service (IHostedService)
- **Phase 10:** Logging & Monitoring (**NEU!**)
- Step 10.1 - Correlation IDs (Request-Tracking)
- Step 10.2 - Serilog Configuration (Seq + Application Insights)
- Step 10.2 - Serilog Configuration (Seq + File Logging)
- **Phase 11:** Production Deployment
- appsettings.Production.json
@@ -2383,35 +2429,44 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
---
### 7. Async Processing: Queue-based (für große Operationen) - **NEU!**
### 7. Async Processing: In-Memory Queue-based (für große Operationen) - **NEU!**
**Entscheidung:** Azure Storage Queue + Background Worker
**Entscheidung:** In-Memory Queue + Background Worker
**Warum:**
- ConcatenatePdfs von 50 PDFs = 10+ Sekunden
- Synchron = HTTP Timeout
- Queue = skalierbar (Worker horizontal skalieren)
- In-Memory Queue = einfach, keine Cloud-Abhängigkeiten
- ProcessingStatus Enum wird jetzt genutzt!
**Alternative wäre gewesen:**
- Alle Operationen synchron
- **Nachteile:** HTTP Timeouts, nicht skalierbar
**Hinweis:**
- In-Memory Queue = nicht persistent (bei Server-Neustart gehen Jobs verloren)
- Für Production später: Redis Queue oder RabbitMQ (aber IJobQueue bleibt!)
- Für Single-Server Setup: In-Memory Queue ausreichend
---
### 8. File Storage: Azure Blob (statt Local Files) - **NEU!**
### 8. File Storage: Lokale Temp-Ordner (statt Cloud) - **NEU!**
**Entscheidung:** IFileStorage Interface + Azure Blob + Local (Dev)
**Entscheidung:** IFileStorage Interface + LocalFileStorage
**Warum:**
- **Multi-Server:** Load Balancer mit 3 API-Instanzen
- **Shared Storage:** Azure Blob = alle Server greifen auf gleiche Files zu
- **Abstraction:** LocalFileStorage für Dev/Test
- **Automatic Cleanup:** TempFileCleanupService (IHostedService)
- **Einfachheit:** Keine Cloud-Abhängigkeiten (Azure Blob Storage)
- **Schnell:** Lokaler Dateizugriff (keine Netzwerk-Latenz)
- **Flexibel:** IFileStorage Abstraction bleibt (später zu Cloud migrierbar!)
- **TempFileCleanupService:** Automatische Cleanup (täglich)
**Alternative wäre gewesen:**
- Temp-Files auf lokalem Server
- **Nachteile:** Multi-Server nicht möglich, Disk voll nach 1 Monat
- Cloud Storage (Azure Blob, AWS S3)
- **Nachteile:** Cloud-Abhängigkeit, Kosten, Komplexität
**Hinweis:**
- Für Multi-Server Setup: Shared Network Drive (UNC-Pfad) statt lokale Ordner
- IFileStorage Interface bleibt gleich (austauschbar!)
---
@@ -2445,19 +2500,19 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
---
### 11. Logging: Correlation IDs + Seq + Application Insights - **NEU!**
### 11. Logging: Correlation IDs + Seq + File Logging - **NEU!**
**Entscheidung:** Structured Logging mit Correlation IDs
**Warum:**
- **Correlation IDs:** Request-Tracking über alle Logs (Debugging leichter)
- **Seq:** Log-Browsing UI (Development)
- **Application Insights:** Production Monitoring (Azure)
- **File Logging:** Production Logs (keine Cloud-Abhängigkeit)
- **LoggingBehavior:** MediatR Pipeline Behavior (automatisches Logging)
**Alternative wäre gewesen:**
- Nur File Logging (keine Correlation IDs)
- **Nachteile:** Debugging schwierig, keine Request-Zusammenhänge
- Nur Console Logging (keine Correlation IDs)
- **Nachteile:** Debugging schwierig, keine Request-Zusammenhänge, keine persistente Log-Speicherung
### 5. Vertical Slice Architecture
@@ -2488,8 +2543,6 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
- [FluentAssertions Documentation](https://fluentassertions.com/)
- [Polly Documentation](https://www.pollydocs.org/) - **NEU:** Resilience Patterns
- [EF Core Documentation](https://learn.microsoft.com/en-us/ef/core/) - **NEU:** ORM für Tenant-DB
- [Azure Blob Storage Documentation](https://learn.microsoft.com/en-us/azure/storage/blobs/) - **NEU:** File Storage
- [Azure Storage Queue Documentation](https://learn.microsoft.com/en-us/azure/storage/queues/) - **NEU:** Async Processing
- [Serilog Documentation](https://serilog.net/) - **NEU:** Structured Logging
- [Seq Documentation](https://docs.datalust.co/docs) - **NEU:** Log Browsing UI
@@ -2504,17 +2557,17 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
- ? TDD (Test-Driven Development)
- ? **Resilience Patterns (Polly)** - **NEU:** Retry, Circuit Breaker, Timeout
- ? **Multi-Tenancy (Database-based)** - **NEU:** EF Core + SQLite + Redis Cache
- ? **Async Processing (Queue-based)** - **NEU:** Azure Storage Queue + Background Worker
- ? **File Storage Abstraction (Azure Blob)** - **NEU:** Multi-Server fähig
- ? **Async Processing (In-Memory Queue)** - **NEU:** In-Memory Queue + Background Worker
- ? **File Storage Abstraction (Lokale Temp-Ordner)** - **NEU:** IFileStorage + LocalFileStorage
- ? **Correlation IDs** - **NEU:** Request-Tracking über alle Logs
- ? **Health Checks** - **NEU:** Kubernetes/Load Balancer Support
- ? **Health Checks** - **NEU:** Load Balancer Support
- ? Options Pattern für Configuration
- ? Dependency Injection
- ? Async/Await überall
- ? Nullable Reference Types
- ? Record Types für DTOs (C# 12)
- ? Primary Constructors (.NET 8)
- ? Structured Logging (Serilog + Seq + Application Insights)
- ? Structured Logging (Serilog + Seq + File Logging)
---
@@ -2540,14 +2593,16 @@ public async Task POST_ValidatePdf_InvalidPdf_Returns400() { }
| 17.01.2025 | Tests | ? Step 3.2.4 - DevExpressPdfProcessorTests.cs erstellt (TDD Red - 6 Tests) |
| 17.01.2025 | Roadmap | ?? **ROADMAP MAJOR UPDATE** - Production-Ready Features hinzugefügt! |
| 17.01.2025 | Architecture | ? **Multi-Tenancy:** Database-based (EF Core + SQLite + Redis Cache) |
| 17.01.2025 | Architecture | ? **Async Processing:** Queue-based (Azure Storage Queue + Background Worker) |
| 17.01.2025 | Architecture | ? **File Storage:** Azure Blob Storage + IFileStorage Abstraction |
| 17.01.2025 | Architecture | ? **Async Processing:** In-Memory Queue-based + Background Worker |
| 17.01.2025 | Architecture | ? **File Storage:** Lokale Temp-Ordner mit IFileStorage Abstraction |
| 17.01.2025 | Architecture | ? **Resilience:** Polly (Retry, Circuit Breaker, Timeout) |
| 17.01.2025 | Architecture | ? **Health Checks:** Früh implementieren (Phase 5.5 statt Phase 9) |
| 17.01.2025 | Architecture | ? **Logging:** Correlation IDs + Seq + Application Insights |
| 17.01.2025 | Technology Stack | ? **11 neue NuGet Packages hinzugefügt** (EF Core, Polly, Azure Storage, BCrypt, Seq) |
| 17.01.2025 | Architecture | ? **Logging:** Correlation IDs + Seq + File Logging |
| 17.01.2025 | Technology Stack | ? **9 neue NuGet Packages hinzugefügt** (EF Core, Polly, BCrypt, Seq - OHNE Azure) |
| 17.01.2025 | Roadmap | ? **6 neue Phasen** (5.5, 6.5, 8, 9, 10, 11) - insgesamt 11 Phasen statt 9 |
| 17.01.2025 | Documentation | ? **11 Key Learnings & Decisions** dokumentiert (statt 5)
| 17.01.2025 | Documentation | ? **10 Key Learnings & Decisions** dokumentiert (statt 5) |
| 17.01.2025 | Roadmap | ?? **Azure Services entfernt** - Lokale Temp-Ordner + In-Memory Queue stattdessen
| 22.06.2026 | Dokumentation | ? **Azure-Referenzen vollständig entfernt** - Alle Azure-Referenzen bereinigt (PROJECT_STATUS.md + ROADMAP.md)
---