Refactor MediatR commands and update solution structure
- Consolidated commands and handlers into single files for better organization. - Updated file naming conventions for commands and queries. - Added explicit Git operation rules to prevent automatic commits/pushes. - Introduced new projects and restructured solution file (`legacy` folder). - Refactored `CreateEmailAccountCommand`, `ProcessEmailCommand`, and others to use `IUnitOfWork`. - Enhanced `ProcessEmailCommandHandler` with attachment validation and error handling. - Removed redundant handler files after consolidation. - Improved code consistency and added `TODO` comments for future enhancements.
This commit is contained in:
448
AGENTS.md
Normal file
448
AGENTS.md
Normal file
@@ -0,0 +1,448 @@
|
||||
# EmailProfiler - Agent Notes and Future Enhancements
|
||||
|
||||
## Purpose
|
||||
This document contains important notes, decisions, and future enhancement plans for the EmailProfiler application. This is intended for AI agents and developers who will continue development.
|
||||
|
||||
---
|
||||
|
||||
## Important Notes
|
||||
|
||||
### 1. Database Schema - DO NOT MODIFY
|
||||
**CRITICAL**: The database schema must NEVER be modified. All Entity Framework entities must map to existing legacy tables using `[Table]` and `[Column]` attributes.
|
||||
|
||||
**Naming Convention**:
|
||||
- Database: `SNAKE_CASE` with prefixes (TBEMLP_, TBDD_)
|
||||
- C# Entities: `PascalCase` without prefixes
|
||||
- Use `[Table("TBDD_FOO")]` and `[Column("COLUMN_NAME")]` attributes
|
||||
|
||||
**Example**:
|
||||
```csharp
|
||||
[Table("TBDD_EMAIL_ACCOUNT")]
|
||||
public class EmailAccount
|
||||
{
|
||||
[Column("EMAIL_ACCOUNT_ID")]
|
||||
public int Id { get; set; }
|
||||
|
||||
[Column("ACCOUNT_NAME")]
|
||||
public string AccountName { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Message ID Hash Algorithm
|
||||
The `MessageIdGenerator` in `Domain.Services` must use **exactly the same algorithm** as the legacy system to ensure duplicate detection works correctly.
|
||||
|
||||
**Algorithm**: SHA256 hash of `{originalMessageId}|{sender}|{date:yyyyMMddHHmmss}|{subject}`
|
||||
|
||||
### 3. DateTime Usage - ALWAYS Use Local Time
|
||||
**CRITICAL**: Always use `DateTime.Now` instead of `DateTime.UtcNow` throughout the entire application.
|
||||
|
||||
**Reason**: The legacy system uses local server time, and the database stores all timestamps as local time. Using UTC would break compatibility and cause incorrect time comparisons.
|
||||
|
||||
**Examples**:
|
||||
```csharp
|
||||
// ✅ CORRECT
|
||||
profile.CreatedDate = DateTime.Now;
|
||||
var lastPoll = DateTime.Now.AddMinutes(-profile.PollIntervalMinutes);
|
||||
|
||||
// ❌ WRONG - DO NOT USE
|
||||
profile.CreatedDate = DateTime.UtcNow; // NEVER USE UTC
|
||||
var lastPoll = DateTime.UtcNow.AddMinutes(-profile.PollIntervalMinutes); // NEVER USE UTC
|
||||
```
|
||||
|
||||
**Important**: This applies to:
|
||||
- All entity audit fields (CreatedDate, ModifiedDate, LastPollDate, etc.)
|
||||
- All date comparisons in business logic
|
||||
- All timestamps in logs and error messages
|
||||
- All date parameters in queries
|
||||
|
||||
### 4. Git Operations - NEVER Without Explicit Permission
|
||||
**CRITICAL**: NEVER execute `git commit` or `git push` commands automatically. ALWAYS wait for explicit user instruction.
|
||||
|
||||
**Rules**:
|
||||
- Only commit when user explicitly says "commit" or "commit this"
|
||||
- Only push when user explicitly says "push" or "push to remote"
|
||||
- Stage files with `git add` ONLY when about to commit per user request
|
||||
|
||||
### 5. MediatR Command/Query File Organization
|
||||
**IMPORTANT**: Commands/Queries and their Handlers must be in the SAME file.
|
||||
|
||||
**Example**:
|
||||
```csharp
|
||||
// ✅ CORRECT - CreateEmailProfileCommand.cs contains BOTH
|
||||
public record CreateEmailProfileCommand : IRequest<int> { ... }
|
||||
|
||||
public class CreateEmailProfileCommandHandler : IRequestHandler<CreateEmailProfileCommand, int> { ... }
|
||||
|
||||
// ❌ WRONG - Separate files
|
||||
// CreateEmailProfileCommand.cs (command only)
|
||||
// CreateEmailProfileCommandHandler.cs (handler only)
|
||||
```
|
||||
|
||||
**File Naming**:
|
||||
- Commands: `{Verb}{Entity}Command.cs` (e.g., `CreateEmailProfileCommand.cs`)
|
||||
- Queries: `{Verb}{Entity}Query.cs` (e.g., `GetEmailProfilesQuery.cs`)
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### HIGH PRIORITY: RabbitMQ Queue Implementation
|
||||
|
||||
**Current State**:
|
||||
- Email queue is implemented using in-memory `Channel<T>` in `InMemoryEmailQueue.cs`
|
||||
- Location: `src/DigitalData.EmailProfiler.Infrastructure/Queue/InMemoryEmailQueue.cs`
|
||||
|
||||
**Future Enhancement**:
|
||||
Replace the in-memory queue with **RabbitMQ** for production resilience and scalability.
|
||||
|
||||
**Implementation Steps**:
|
||||
|
||||
1. **Add NuGet Package**:
|
||||
```bash
|
||||
dotnet add package RabbitMQ.Client
|
||||
```
|
||||
|
||||
2. **Create RabbitMqEmailQueue.cs**:
|
||||
```csharp
|
||||
// src/DigitalData.EmailProfiler.Infrastructure/Queue/RabbitMqEmailQueue.cs
|
||||
public class RabbitMqEmailQueue : IEmailQueue
|
||||
{
|
||||
private readonly IConnection _connection;
|
||||
private readonly IModel _channel;
|
||||
private const string QueueName = "email-outbox";
|
||||
|
||||
public RabbitMqEmailQueue(IOptions<RabbitMqConfiguration> config)
|
||||
{
|
||||
var factory = new ConnectionFactory
|
||||
{
|
||||
HostName = config.Value.HostName,
|
||||
Port = config.Value.Port,
|
||||
UserName = config.Value.UserName,
|
||||
Password = config.Value.Password
|
||||
};
|
||||
|
||||
_connection = factory.CreateConnection();
|
||||
_channel = _connection.CreateModel();
|
||||
|
||||
_channel.QueueDeclare(
|
||||
queue: QueueName,
|
||||
durable: true,
|
||||
exclusive: false,
|
||||
autoDelete: false,
|
||||
arguments: null);
|
||||
}
|
||||
|
||||
public async Task EnqueueAsync(OutgoingEmail email, CancellationToken cancellationToken)
|
||||
{
|
||||
var json = JsonSerializer.Serialize(email);
|
||||
var body = Encoding.UTF8.GetBytes(json);
|
||||
|
||||
var properties = _channel.CreateBasicProperties();
|
||||
properties.Persistent = true;
|
||||
|
||||
_channel.BasicPublish(
|
||||
exchange: "",
|
||||
routingKey: QueueName,
|
||||
basicProperties: properties,
|
||||
body: body);
|
||||
|
||||
await Task.CompletedTask;
|
||||
}
|
||||
|
||||
public async Task<OutgoingEmail?> DequeueAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
var result = _channel.BasicGet(QueueName, autoAck: false);
|
||||
|
||||
if (result == null)
|
||||
return null;
|
||||
|
||||
var json = Encoding.UTF8.GetString(result.Body.ToArray());
|
||||
var email = JsonSerializer.Deserialize<OutgoingEmail>(json);
|
||||
|
||||
_channel.BasicAck(result.DeliveryTag, false);
|
||||
|
||||
return await Task.FromResult(email);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **Configuration** (appsettings.json):
|
||||
```json
|
||||
{
|
||||
"RabbitMq": {
|
||||
"HostName": "localhost",
|
||||
"Port": 5672,
|
||||
"UserName": "guest",
|
||||
"Password": "guest"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
4. **Dependency Injection** (Program.cs):
|
||||
```csharp
|
||||
// Replace InMemoryEmailQueue with RabbitMqEmailQueue
|
||||
// builder.Services.AddSingleton<IEmailQueue, InMemoryEmailQueue>();
|
||||
builder.Services.AddSingleton<IEmailQueue, RabbitMqEmailQueue>();
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Message persistence (survives application restarts)
|
||||
- Scalability (multiple worker instances can consume from queue)
|
||||
- Reliability (automatic retries, dead letter queues)
|
||||
- Monitoring (RabbitMQ management UI)
|
||||
|
||||
**Migration Path**:
|
||||
1. Deploy RabbitMQ server (Docker recommended)
|
||||
2. Test RabbitMqEmailQueue in staging environment
|
||||
3. Switch DI registration from InMemoryEmailQueue to RabbitMqEmailQueue
|
||||
4. Monitor queue depth and worker performance
|
||||
|
||||
---
|
||||
|
||||
## Pending Implementation Tasks
|
||||
|
||||
### Phase 2: Application Layer (IN PROGRESS)
|
||||
**Status**: Partially complete - DTOs created, Commands/Queries needed
|
||||
|
||||
**TODO**:
|
||||
- [ ] Create MediatR Commands (CreateEmailProfileCommand, ProcessEmailCommand, etc.)
|
||||
- [ ] Create MediatR Queries (GetEmailProfilesQuery, GetEmailHistoryQuery, etc.)
|
||||
- [ ] Create Command/Query Handlers
|
||||
- [ ] Create FluentValidation Validators
|
||||
- [ ] Create AutoMapper Profiles
|
||||
- [ ] Create Application Interfaces (IEmailService, IPdfProcessingService, IDmsService, etc.)
|
||||
|
||||
**Example Command**:
|
||||
```csharp
|
||||
// src/DigitalData.EmailProfiler.Application/EmailProfiles/Commands/CreateEmailProfileCommand.cs
|
||||
public record CreateEmailProfileCommand(string ProfileName, int EmailAccountId) : IRequest<int>;
|
||||
|
||||
public class CreateEmailProfileCommandHandler : IRequestHandler<CreateEmailProfileCommand, int>
|
||||
{
|
||||
private readonly IEmailProfileRepository _repository;
|
||||
|
||||
public async Task<int> Handle(CreateEmailProfileCommand request, CancellationToken cancellationToken)
|
||||
{
|
||||
var profile = new EmailProfile
|
||||
{
|
||||
ProfileName = request.ProfileName,
|
||||
EmailAccountId = request.EmailAccountId,
|
||||
IsActive = true
|
||||
};
|
||||
|
||||
await _repository.AddAsync(profile, cancellationToken);
|
||||
return profile.Id;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 3: Infrastructure Layer
|
||||
**Status**: Not started
|
||||
|
||||
**TODO**:
|
||||
- [ ] Create EmailProfilerDbContext with DbSet<T> for all entities
|
||||
- [ ] Create Entity Configurations (Fluent API) for all entities
|
||||
- [ ] Create Repositories implementing Application interfaces
|
||||
- [ ] Create MailKitEmailService (IMAP/SMTP with OAuth2)
|
||||
- [ ] Create PdfSharpProcessingService
|
||||
- [ ] Create WindreamDmsService (COM Interop)
|
||||
- [ ] Create EncryptionService (Data Protection API)
|
||||
- [ ] Create initial EF Core migration
|
||||
|
||||
**DbContext Example**:
|
||||
```csharp
|
||||
public class EmailProfilerDbContext : DbContext
|
||||
{
|
||||
public DbSet<EmailAccount> EmailAccounts { get; set; }
|
||||
public DbSet<EmailProfile> EmailProfiles { get; set; }
|
||||
// ... other DbSets
|
||||
|
||||
protected override void OnModelCreating(ModelBuilder modelBuilder)
|
||||
{
|
||||
modelBuilder.ApplyConfigurationsFromAssembly(Assembly.GetExecutingAssembly());
|
||||
|
||||
// Important: Check for triggers
|
||||
modelBuilder.Entity<EmailHistory>().ToTable(tb => tb.HasTrigger("TR_TBEMLP_HISTORY_AUDIT"));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 4: API Layer
|
||||
**Status**: Minimal structure exists
|
||||
|
||||
**TODO**:
|
||||
- [ ] Create Controllers (ProfilesController, EmailAccountsController, HistoryController)
|
||||
- [ ] Create Background Workers (EmailPollingWorker, EmailSenderWorker)
|
||||
- [ ] Configure Serilog
|
||||
- [ ] Configure Scalar (OpenAPI documentation)
|
||||
- [ ] Add Exception Handling Middleware
|
||||
- [ ] Configure DI for all layers
|
||||
- [ ] Support both IIS and Windows Service hosting
|
||||
|
||||
**Worker Configuration** (appsettings.json):
|
||||
```json
|
||||
{
|
||||
"Workers": {
|
||||
"EmailPolling": {
|
||||
"Enabled": true,
|
||||
"IntervalSeconds": 60
|
||||
},
|
||||
"EmailSender": {
|
||||
"Enabled": true,
|
||||
"IntervalSeconds": 5
|
||||
}
|
||||
},
|
||||
"Hosting": {
|
||||
"Mode": "IIS" // or "WindowsService"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 5: Testing
|
||||
**Status**: Not started
|
||||
|
||||
**TODO**:
|
||||
- [ ] Unit tests for Domain entities
|
||||
- [ ] Unit tests for Application handlers (using FakeItEasy)
|
||||
- [ ] Integration tests for Repositories (using Testcontainers)
|
||||
- [ ] API tests (using WebApplicationFactory)
|
||||
- [ ] Generate fake test data (using Bogus)
|
||||
|
||||
**Test Example**:
|
||||
```csharp
|
||||
public class MessageIdGeneratorTests
|
||||
{
|
||||
[Fact]
|
||||
public void Generate_ShouldProduceSameHashAsLegacy()
|
||||
{
|
||||
// Arrange
|
||||
var generator = new MessageIdGenerator();
|
||||
var original = "msg-123";
|
||||
var sender = "test@example.com";
|
||||
var date = new DateTime(2026, 1, 1, 12, 0, 0);
|
||||
var subject = "Test Subject";
|
||||
|
||||
// Act
|
||||
var messageId = generator.Generate(original, sender, date, subject);
|
||||
|
||||
// Assert
|
||||
messageId.Hash.Should().NotBeNullOrEmpty();
|
||||
// TODO: Verify against known legacy hash
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
### Clean Architecture Layers
|
||||
1. **Domain**: Core business logic, no dependencies
|
||||
2. **Application**: Use cases, depends on Domain
|
||||
3. **Infrastructure**: External concerns, depends on Domain + Application
|
||||
4. **API**: Entry point, depends on all
|
||||
|
||||
### CQRS Pattern with MediatR
|
||||
- **Commands**: Modify state (Create, Update, Delete)
|
||||
- **Queries**: Read data (Get, List)
|
||||
- Separate models for read and write operations
|
||||
|
||||
### Repository Pattern
|
||||
- Interface in Application layer
|
||||
- Implementation in Infrastructure layer
|
||||
- One repository per Aggregate Root
|
||||
|
||||
---
|
||||
|
||||
## Known Issues and Limitations
|
||||
|
||||
### 1. PdfSharp Embedded File Extraction
|
||||
PdfSharp has limited support for embedded file extraction from PDFs. If advanced PDF processing is needed, consider:
|
||||
- **iText7** (AGPL or commercial license)
|
||||
- **Aspose.PDF** (commercial license)
|
||||
- Custom PDF parsing using PDF specification
|
||||
|
||||
### 2. windream COM Interop
|
||||
The windream DMS integration uses COM Interop which is Windows-only. The application cannot be fully cross-platform unless windream provides a REST API alternative.
|
||||
|
||||
### 3. OAuth2 Token Refresh
|
||||
Current implementation acquires new tokens on each request. Consider implementing token caching:
|
||||
- Use `Microsoft.Identity.Web` for automatic token management
|
||||
- Cache tokens in memory or distributed cache (Redis)
|
||||
|
||||
---
|
||||
|
||||
## Development Guidelines
|
||||
|
||||
### 1. Code Style
|
||||
- All code and comments: **English**
|
||||
- README.md and user documentation: **German**
|
||||
- Follow C# naming conventions (PascalCase, camelCase)
|
||||
- Use nullable reference types (`#nullable enable`)
|
||||
|
||||
### 2. Logging
|
||||
Use Serilog with structured logging:
|
||||
```csharp
|
||||
_logger.LogInformation("Processing email {MessageId} from profile {ProfileId}", messageId, profileId);
|
||||
```
|
||||
|
||||
### 3. Configuration
|
||||
- Development: `appsettings.Development.json` + User Secrets
|
||||
- Production: `appsettings.json` + Environment Variables + Azure Key Vault
|
||||
|
||||
### 4. Error Handling
|
||||
- Domain: Throw `DomainException` for business rule violations
|
||||
- Application: Use `FluentValidation` for input validation
|
||||
- API: Use exception handling middleware to return proper HTTP status codes
|
||||
|
||||
---
|
||||
|
||||
## Deployment Scenarios
|
||||
|
||||
### IIS Hosting (Default)
|
||||
```json
|
||||
{
|
||||
"Hosting": {
|
||||
"Mode": "IIS"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Windows Service Hosting
|
||||
```json
|
||||
{
|
||||
"Hosting": {
|
||||
"Mode": "WindowsService"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
In `Program.cs`:
|
||||
```csharp
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
|
||||
if (builder.Configuration["Hosting:Mode"] == "WindowsService")
|
||||
{
|
||||
builder.Host.UseWindowsService();
|
||||
}
|
||||
```
|
||||
|
||||
Install as Windows Service:
|
||||
```bash
|
||||
sc create EmailProfiler binPath="C:\Path\To\DigitalData.EmailProfiler.API.exe"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contact and Support
|
||||
|
||||
For questions about this implementation, consult:
|
||||
- Legacy system analysis: `legacy/PROJECT_ANALYSIS.md`
|
||||
- Migration plan: `MIGRATION_PLAN.md` (if created)
|
||||
- This document: `agents.md`
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2026-07-07
|
||||
**Version**: 1.0
|
||||
**Status**: Phase 1 Complete (Domain Layer), Phase 2-8 Pending
|
||||
Reference in New Issue
Block a user