AppStd/DigitalData.MessagingService
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add critical notes and future enhancements for agents

Browse Source 
Important notes:
- Database schema must NEVER be modified
- MessageId hash algorithm must match legacy system exactly
- No git commits without explicit permission
- Naming conventions (SNAKE_CASE DB, PascalCase C#)

Future enhancements:
- RabbitMQ queue implementation plan (replacing in-memory queue)
- Complete migration path and configuration examples
- Pending implementation tasks for each phase
- Known issues and limitations (PdfSharp, windream COM)

Architecture decisions:
- Clean Architecture with DDD
- CQRS pattern with MediatR
- Repository pattern

Development guidelines:
- Code style conventions
- Logging with Serilog
- Configuration management
- Error handling strategies
- Deployment scenarios (IIS/Windows Service)
This commit is contained in:
tekh
2026-07-07 18:59:57 +02:00
parent e789afe26a
commit 331b73000e
1 changed files with 402 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

402
agents.md Normal file
View File

@@ -0,0 +1,402 @@
# EmailProfiler - Agent Notes and Future Enhancements
## Purpose
This document contains important notes, decisions, and future enhancement plans for the EmailProfiler application. This is intended for AI agents and developers who will continue development.
---
## Important Notes
### 1. Database Schema - DO NOT MODIFY
**CRITICAL**: The database schema must NEVER be modified. All Entity Framework entities must map to existing legacy tables using `[Table]` and `[Column]` attributes.
**Naming Convention**:
- Database: `SNAKE_CASE` with prefixes (TBEMLP_, TBDD_)
- C# Entities: `PascalCase` without prefixes
- Use `[Table("TBDD_FOO")]` and `[Column("COLUMN_NAME")]` attributes
**Example**:
```csharp
[Table("TBDD_EMAIL_ACCOUNT")]
public class EmailAccount
{
[Column("EMAIL_ACCOUNT_ID")]
public int Id { get; set; }
[Column("ACCOUNT_NAME")]
public string AccountName { get; set; }
}
```
### 2. Message ID Hash Algorithm
The `MessageIdGenerator` in `Domain.Services` must use **exactly the same algorithm** as the legacy system to ensure duplicate detection works correctly.
**Algorithm**: SHA256 hash of `{originalMessageId}|{sender}|{date:yyyyMMddHHmmss}|{subject}`
### 3. No Commits Without Permission
**NEVER** commit changes to git automatically. Always wait for explicit user instruction to commit.
---
## Future Enhancements
### HIGH PRIORITY: RabbitMQ Queue Implementation
**Current State**:
- Email queue is implemented using in-memory `Channel<T>` in `InMemoryEmailQueue.cs`
- Location: `src/DigitalData.EmailProfiler.Infrastructure/Queue/InMemoryEmailQueue.cs`
**Future Enhancement**:
Replace the in-memory queue with **RabbitMQ** for production resilience and scalability.
**Implementation Steps**:
1. **Add NuGet Package**:
```bash
dotnet add package RabbitMQ.Client
```
2. **Create RabbitMqEmailQueue.cs**:
```csharp
// src/DigitalData.EmailProfiler.Infrastructure/Queue/RabbitMqEmailQueue.cs
public class RabbitMqEmailQueue : IEmailQueue
{
private readonly IConnection _connection;
private readonly IModel _channel;
private const string QueueName = "email-outbox";
public RabbitMqEmailQueue(IOptions<RabbitMqConfiguration> config)
{
var factory = new ConnectionFactory
{
HostName = config.Value.HostName,
Port = config.Value.Port,
UserName = config.Value.UserName,
Password = config.Value.Password
};
_connection = factory.CreateConnection();
_channel = _connection.CreateModel();
_channel.QueueDeclare(
queue: QueueName,
durable: true,
exclusive: false,
autoDelete: false,
arguments: null);
}
public async Task EnqueueAsync(OutgoingEmail email, CancellationToken cancellationToken)
{
var json = JsonSerializer.Serialize(email);
var body = Encoding.UTF8.GetBytes(json);
var properties = _channel.CreateBasicProperties();
properties.Persistent = true;
_channel.BasicPublish(
exchange: "",
routingKey: QueueName,
basicProperties: properties,
body: body);
await Task.CompletedTask;
}
public async Task<OutgoingEmail?> DequeueAsync(CancellationToken cancellationToken)
{
var result = _channel.BasicGet(QueueName, autoAck: false);
if (result == null)
return null;
var json = Encoding.UTF8.GetString(result.Body.ToArray());
var email = JsonSerializer.Deserialize<OutgoingEmail>(json);
_channel.BasicAck(result.DeliveryTag, false);
return await Task.FromResult(email);
}
}
```
3. **Configuration** (appsettings.json):
```json
{
"RabbitMq": {
"HostName": "localhost",
"Port": 5672,
"UserName": "guest",
"Password": "guest"
}
}
```
4. **Dependency Injection** (Program.cs):
```csharp
// Replace InMemoryEmailQueue with RabbitMqEmailQueue
// builder.Services.AddSingleton<IEmailQueue, InMemoryEmailQueue>();
builder.Services.AddSingleton<IEmailQueue, RabbitMqEmailQueue>();
```
**Benefits**:
- Message persistence (survives application restarts)
- Scalability (multiple worker instances can consume from queue)
- Reliability (automatic retries, dead letter queues)
- Monitoring (RabbitMQ management UI)
**Migration Path**:
1. Deploy RabbitMQ server (Docker recommended)
2. Test RabbitMqEmailQueue in staging environment
3. Switch DI registration from InMemoryEmailQueue to RabbitMqEmailQueue
4. Monitor queue depth and worker performance
---
## Pending Implementation Tasks
### Phase 2: Application Layer (IN PROGRESS)
**Status**: Partially complete - DTOs created, Commands/Queries needed
**TODO**:
- [ ] Create MediatR Commands (CreateEmailProfileCommand, ProcessEmailCommand, etc.)
- [ ] Create MediatR Queries (GetEmailProfilesQuery, GetEmailHistoryQuery, etc.)
- [ ] Create Command/Query Handlers
- [ ] Create FluentValidation Validators
- [ ] Create AutoMapper Profiles
- [ ] Create Application Interfaces (IEmailService, IPdfProcessingService, IDmsService, etc.)
**Example Command**:
```csharp
// src/DigitalData.EmailProfiler.Application/EmailProfiles/Commands/CreateEmailProfileCommand.cs
public record CreateEmailProfileCommand(string ProfileName, int EmailAccountId) : IRequest<int>;
public class CreateEmailProfileCommandHandler : IRequestHandler<CreateEmailProfileCommand, int>
{
private readonly IEmailProfileRepository _repository;
public async Task<int> Handle(CreateEmailProfileCommand request, CancellationToken cancellationToken)
{
var profile = new EmailProfile
{
ProfileName = request.ProfileName,
EmailAccountId = request.EmailAccountId,
IsActive = true
};
await _repository.AddAsync(profile, cancellationToken);
return profile.Id;
}
}
```
### Phase 3: Infrastructure Layer
**Status**: Not started
**TODO**:
- [ ] Create EmailProfilerDbContext with DbSet<T> for all entities
- [ ] Create Entity Configurations (Fluent API) for all entities
- [ ] Create Repositories implementing Application interfaces
- [ ] Create MailKitEmailService (IMAP/SMTP with OAuth2)
- [ ] Create PdfSharpProcessingService
- [ ] Create WindreamDmsService (COM Interop)
- [ ] Create EncryptionService (Data Protection API)
- [ ] Create initial EF Core migration
**DbContext Example**:
```csharp
public class EmailProfilerDbContext : DbContext
{
public DbSet<EmailAccount> EmailAccounts { get; set; }
public DbSet<EmailProfile> EmailProfiles { get; set; }
// ... other DbSets
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.ApplyConfigurationsFromAssembly(Assembly.GetExecutingAssembly());
// Important: Check for triggers
modelBuilder.Entity<EmailHistory>().ToTable(tb => tb.HasTrigger("TR_TBEMLP_HISTORY_AUDIT"));
}
}
```
### Phase 4: API Layer
**Status**: Minimal structure exists
**TODO**:
- [ ] Create Controllers (ProfilesController, EmailAccountsController, HistoryController)
- [ ] Create Background Workers (EmailPollingWorker, EmailSenderWorker)
- [ ] Configure Serilog
- [ ] Configure Scalar (OpenAPI documentation)
- [ ] Add Exception Handling Middleware
- [ ] Configure DI for all layers
- [ ] Support both IIS and Windows Service hosting
**Worker Configuration** (appsettings.json):
```json
{
"Workers": {
"EmailPolling": {
"Enabled": true,
"IntervalSeconds": 60
},
"EmailSender": {
"Enabled": true,
"IntervalSeconds": 5
}
},
"Hosting": {
"Mode": "IIS" // or "WindowsService"
}
}
```
### Phase 5: Testing
**Status**: Not started
**TODO**:
- [ ] Unit tests for Domain entities
- [ ] Unit tests for Application handlers (using FakeItEasy)
- [ ] Integration tests for Repositories (using Testcontainers)
- [ ] API tests (using WebApplicationFactory)
- [ ] Generate fake test data (using Bogus)
**Test Example**:
```csharp
public class MessageIdGeneratorTests
{
[Fact]
public void Generate_ShouldProduceSameHashAsLegacy()
{
// Arrange
var generator = new MessageIdGenerator();
var original = "msg-123";
var sender = "test@example.com";
var date = new DateTime(2026, 1, 1, 12, 0, 0);
var subject = "Test Subject";
// Act
var messageId = generator.Generate(original, sender, date, subject);
// Assert
messageId.Hash.Should().NotBeNullOrEmpty();
// TODO: Verify against known legacy hash
}
}
```
---
## Architecture Decisions
### Clean Architecture Layers
1. **Domain**: Core business logic, no dependencies
2. **Application**: Use cases, depends on Domain
3. **Infrastructure**: External concerns, depends on Domain + Application
4. **API**: Entry point, depends on all
### CQRS Pattern with MediatR
- **Commands**: Modify state (Create, Update, Delete)
- **Queries**: Read data (Get, List)
- Separate models for read and write operations
### Repository Pattern
- Interface in Application layer
- Implementation in Infrastructure layer
- One repository per Aggregate Root
---
## Known Issues and Limitations
### 1. PdfSharp Embedded File Extraction
PdfSharp has limited support for embedded file extraction from PDFs. If advanced PDF processing is needed, consider:
- **iText7** (AGPL or commercial license)
- **Aspose.PDF** (commercial license)
- Custom PDF parsing using PDF specification
### 2. windream COM Interop
The windream DMS integration uses COM Interop which is Windows-only. The application cannot be fully cross-platform unless windream provides a REST API alternative.
### 3. OAuth2 Token Refresh
Current implementation acquires new tokens on each request. Consider implementing token caching:
- Use `Microsoft.Identity.Web` for automatic token management
- Cache tokens in memory or distributed cache (Redis)
---
## Development Guidelines
### 1. Code Style
- All code and comments: **English**
- README.md and user documentation: **German**
- Follow C# naming conventions (PascalCase, camelCase)
- Use nullable reference types (`#nullable enable`)
### 2. Logging
Use Serilog with structured logging:
```csharp
_logger.LogInformation("Processing email {MessageId} from profile {ProfileId}", messageId, profileId);
```
### 3. Configuration
- Development: `appsettings.Development.json` + User Secrets
- Production: `appsettings.json` + Environment Variables + Azure Key Vault
### 4. Error Handling
- Domain: Throw `DomainException` for business rule violations
- Application: Use `FluentValidation` for input validation
- API: Use exception handling middleware to return proper HTTP status codes
---
## Deployment Scenarios
### IIS Hosting (Default)
```json
{
"Hosting": {
"Mode": "IIS"
}
}
```
### Windows Service Hosting
```json
{
"Hosting": {
"Mode": "WindowsService"
}
}
```
In `Program.cs`:
```csharp
var builder = WebApplication.CreateBuilder(args);
if (builder.Configuration["Hosting:Mode"] == "WindowsService")
{
builder.Host.UseWindowsService();
}
```
Install as Windows Service:
```bash
sc create EmailProfiler binPath="C:\Path\To\DigitalData.EmailProfiler.API.exe"
```
---
## Contact and Support
For questions about this implementation, consult:
- Legacy system analysis: `legacy/PROJECT_ANALYSIS.md`
- Migration plan: `MIGRATION_PLAN.md` (if created)
- This document: `agents.md`
---
**Last Updated**: 2026-07-07
**Version**: 1.0
**Status**: Phase 1 Complete (Domain Layer), Phase 2-8 Pending