Commit Graph

138 Commits

Author SHA1 Message Date
27513e73f5 feat(application): Add EmailProcessedEvent domain event 2026-07-23 11:20:14 +02:00
658040bd96 refactor(api): Update EmailsController and EmailSenderWorker for simplified email sending 2026-07-23 11:20:09 +02:00
0adc74e19f refactor(infrastructure): Update RabbitMqCommandConsumer with improved error handling 2026-07-23 11:20:04 +02:00
3be6e28477 refactor(infrastructure): Refactor RabbitMqEmailQueue and remove InMemoryEmailQueue, update DbContext 2026-07-23 11:19:59 +02:00
71e29ac3bb refactor(application): Simplify SendEmailCommand and IEmailQueue/IEmailService interfaces, add Shared reference 2026-07-23 11:19:54 +02:00
828bb168eb refactor(application): Remove old DTOs and mapping profiles, consolidate into EmailMappingProfile 2026-07-23 11:19:47 +02:00
70dd210555 refactor(application): Remove old repository and service interfaces 2026-07-23 11:19:41 +02:00
860ce41192 refactor(domain): Remove entity and event files, add DigitalData.EmailProfiler.Shared reference 2026-07-23 11:19:36 +02:00
1404f90729 remove default Worker 2026-07-22 11:57:53 +02:00
958352a720 feat: Add domain constants, API infrastructure, and configuration
Domain Layer:
- Add DomainConstants for email, attachment, and process constants

API Layer:
- Add EmailsController (minimal REST API endpoints)
- Add ExceptionHandlingMiddleware for global exception handling
- Update Program.cs:
  * Add EmailProfilerDbContext registration (SQL Server)
  * Add Generic Repository<T> scoped registration
  * Add ExceptionHandlingMiddleware to pipeline
  * Add EmailSenderWorker as hosted service
  * Configure Serilog file logging
  * Add Scalar OpenAPI documentation

Configuration:
- Add EmailAccount section in appsettings.json (SMTP credentials)
- Add RabbitMq section (message queue configuration)
- Add Serilog file sink configuration
- Update .csproj with required NuGet packages
- Update solution file

This commit completes the basic API infrastructure setup.
2026-07-22 11:50:17 +02:00
8003715792 feat: Add email sending feature with background worker
Application Layer:
- Add SendEmailCommand with handler (CQRS pattern)
- Add SendEmailCommandValidator (FluentValidation)
- Add EmailOutboxMappingProfile for EmailOutbox entity mappings
- Update EmailAccountMappingProfile with latest field mappings
- Update EmailProfileMappingProfile with latest field mappings

API Layer:
- Add EmailSenderWorker background service
- Worker polls EmailOutbox queue every 5 seconds
- Dequeues emails and processes via SendEmailCommand (MediatR)
- Uses IEmailService (Limilabs) for actual SMTP sending

This implements the outgoing email queue processing pipeline:
EmailOutbox (DB) → IEmailQueue (RabbitMQ) → EmailSenderWorker → SendEmailCommand → IEmailService
2026-07-22 11:49:57 +02:00
d346ed3176 refactor(application): Remove old CQRS commands/queries for minimal API migration
- Remove EmailAccounts CQRS layer (4 files: commands, queries, validators)
- Remove EmailHistories CQRS layer (2 files: queries)
- Remove EmailProcessing CQRS layer (2 files: commands, validators)
- Remove EmailProfiles CQRS layer (8 files: commands, queries, validators)
- Remove corresponding API controllers (3 files)

Total: 19 files removed

Reason: Migrating from full CQRS pattern to minimal API with direct repository access
Note: SendEmailCommand will be added separately for EmailSenderWorker
2026-07-22 11:49:37 +02:00
3f9bfc78a8 feat(infrastructure): Add Limilabs email service and RabbitMQ email queue
- Add LimilabsEmailService for SMTP operations (IEmailService implementation)
- Add RabbitMqEmailQueue for production email queue (RabbitMQ-based)
- Update InMemoryEmailQueue for improved error handling
- Update IEmailQueue interface for RabbitMQ compatibility
- Update DependencyInjection.cs:
  * Switch IEmailService to LimilabsEmailService (Singleton)
  * Switch IEmailQueue to RabbitMqEmailQueue (Singleton)
  * Change IEncryptionService to Singleton (thread-safe)
  * Remove IDmsService registration
- Add required NuGet package references to .csproj

TODO: Add Limilabs.Mail NuGet package (commercial license required)
2026-07-22 11:48:37 +02:00
dbd0d35ba3 refactor: Remove MailKit and windream DMS dependencies
- Remove IDmsService interface (DMS integration deferred to Phase 5)
- Remove MailKitEmailService implementation
- Remove WindreamDmsService implementation
- Simplify IEmailService to SMTP-only operations
- Preparing for Limilabs Mail.dll migration

BREAKING CHANGE: IEmailService no longer supports IMAP/POP3 operations
Reason: Migrating from MailKit to Limilabs Mail.dll
2026-07-22 11:48:19 +02:00
751ef87506 refactor(infrastructure): Improve service implementations and remove legacy references
**Services Refactored:**
- DevExpressPdfProcessingService: Remove unnecessary try-catch (lines 80-87), add stream position validation
- WindreamDmsService: Mark as [Obsolete] - application now only provides email sending functionality
- MailKitEmailService: Keep MailKit implementation (Limilabs DLL to be added separately)

**Custom Exceptions Added:**
- AuthenticationFailedException: OAuth2/IMAP/SMTP authentication failures
- DmsNotAvailableException: windream COM unavailable
- InvalidPdfException: Invalid PDF stream
- NotFoundException: Entity not found in Repository operations

**Legacy Cleanup:**
- Remove legacy VB.NET projects from solution (EmailProfiler.Common, EmailProfiler.Service)
- Delete legacy/ folder reference
- Clean solution file structure

**Stream Validation:**
- All PDF processing methods now validate stream position (reset to 0 if needed)
- Add CanSeek validation for stream-based operations

**Build Status:**  Successful (0 errors, 15 warnings - all acceptable)
2026-07-20 16:36:17 +02:00
8f2365d048 remove Features-directory and move the files to the root directory 2026-07-15 15:03:12 +02:00
bfe24eba06 feat(api): Add REST API controllers with RabbitMQ for POST/PUT/DELETE
Controllers:
- EmailProfilesController: CRUD operations (GET sync, POST/PUT/DELETE async via RabbitMQ)
  * GET /api/emailprofiles - List all profiles
  * GET /api/emailprofiles/{id} - Get profile by ID
  * GET /api/emailprofiles/active - List active profiles
  * POST /api/emailprofiles - Create (202 Accepted, queued to RabbitMQ)
  * PUT /api/emailprofiles/{id} - Update (202 Accepted, queued to RabbitMQ)
  * DELETE /api/emailprofiles/{id} - Delete (202 Accepted, queued to RabbitMQ)

- EmailAccountsController: CRUD operations
  * GET /api/emailaccounts - List all accounts
  * GET /api/emailaccounts/{id} - Get account by ID
  * POST /api/emailaccounts - Create (202 Accepted, queued to RabbitMQ)

- EmailHistoryController: Read-only operations
  * GET /api/emailhistory/profile/{profileId} - Get history with pagination
  * GET /api/emailhistory/{id} - Get history by ID

Changes:
- Fix ICommandPublisher constraint: IRequest → IBaseRequest (supports IRequest<T>)
- All POST/PUT/DELETE return HTTP 202 Accepted (async processing)
- All GET operations return HTTP 200 OK (synchronous via MediatR)
- Proper error handling: 404 Not Found for missing resources
2026-07-14 16:39:35 +02:00
1ed489532d feat(application): Add AutoMapper profiles for all entities
- EmailProfileMappingProfile: Command→Entity, DTO→Entity, Entity→DTO
- EmailAccountMappingProfile: Command→Entity, Entity→DTO
- EmailHistoryMappingProfile: CreateDto→Entity, UpdateDto partial mapping, Entity→DTO
- EmailAttachmentMappingProfile: CreateDto→Entity, UpdateDto partial mapping, Entity→DTO

All mappings follow Repository<T> pattern with AutoMapper-based CRUD
2026-07-14 16:37:11 +02:00
eda6257145 feat(application): Add MediatR Commands, Queries, and FluentValidation
Commands (5):
- CreateEmailProfileCommand, UpdateEmailProfileCommand, DeleteEmailProfileCommand
- CreateEmailAccountCommand (OAuth2/password conditional validation)
- ProcessEmailCommand (with CreateEmailHistoryDto, UpdateEmailHistoryStatusDto)

Queries (7):
- GetEmailProfilesQuery, GetEmailProfileByIdQuery, GetActiveEmailProfilesQuery
- GetEmailAccountsQuery, GetEmailAccountByIdQuery
- GetEmailHistoryByProfileQuery (with pagination), GetEmailHistoryByIdQuery

Validators (4):
- CreateEmailProfileCommandValidator, UpdateEmailProfileCommandValidator
- CreateEmailAccountCommandValidator, ProcessEmailCommandValidator

All handlers in same file as commands/queries (AGENTS.md rule #5)
2026-07-14 16:37:00 +02:00
a708799587 refactor(application): Reorganize DTOs - flatten single-DTO folders
- Move EmailAccountDto.cs to Dtos/ (was in EmailAccounts/ subfolder)
- Move EmailProfileDto.cs to Dtos/ (was in EmailProfiles/ subfolder)
- Keep EmailAttachments/ (3 DTOs) and EmailHistories/ (3 DTOs) subfolders
- Update all namespace imports from Dtos.EmailAccounts/EmailProfiles to Dtos
- Simpler structure: single DTOs at root, multiple DTOs in subfolders
2026-07-14 16:36:50 +02:00
b7d65d7d5c refactor(application): Remove IUnitOfWork, add generic IRepository<T> with AutoMapper
- Remove IUnitOfWork pattern (not needed with auto-save repositories)
- Add generic IRepository<T> with CreateAsync<TDto>, UpdateSingleAsync<TDto>, DeleteSingleAsync
- Add UpdateAsync/DeleteAsync for bulk operations
- Add ICommandPublisher interface for RabbitMQ integration
- Add service interfaces: IEmailService, IPdfProcessingService, IDmsService, IEncryptionService, IEmailQueue
- Configure Application DI with MediatR, AutoMapper, FluentValidation
2026-07-14 16:36:41 +02:00
5e8e6a06fe feat(infrastructure): Add RabbitMQ Command Bus integration
- Add RabbitMQ.Client 7.2.1, Microsoft.Extensions.Hosting 10.0.9
- Implement ICommandPublisher interface for async command publishing
- Create RabbitMqCommandPublisher with persistent message delivery
- Create RabbitMqCommandConsumer BackgroundService for command processing
- Add RabbitMqConfiguration with appsettings.json binding
- Configure Infrastructure DI with RabbitMQ services
- Update Program.cs to support appsettings.Secrets.json
- Server: 172.24.12.56:5672, Exchange: emailprofiler.commands
2026-07-14 16:36:23 +02:00
50c21ee628 Refactor MediatR commands and update solution structure
- Consolidated commands and handlers into single files for better organization.
- Updated file naming conventions for commands and queries.
- Added explicit Git operation rules to prevent automatic commits/pushes.
- Introduced new projects and restructured solution file (`legacy` folder).
- Refactored `CreateEmailAccountCommand`, `ProcessEmailCommand`, and others to use `IUnitOfWork`.
- Enhanced `ProcessEmailCommandHandler` with attachment validation and error handling.
- Removed redundant handler files after consolidation.
- Improved code consistency and added `TODO` comments for future enhancements.
2026-07-09 14:37:12 +02:00
45654796b7 feat(application): add MediatR commands and handlers with exception improvements
MediatR Commands (CQRS Pattern):
- CreateEmailProfileCommand + Handler
- UpdateEmailProfileCommand + Handler
- DeleteEmailProfileCommand + Handler
- CreateEmailAccountCommand + Handler
- ProcessEmailCommand + Handler (core email processing logic)

Command Handlers:
- Create/Update/Delete operations for EmailProfile
- Create operation for EmailAccount
- ProcessEmail: Complete email processing workflow including:
  * Duplicate detection using MessageId hash
  * Email history creation
  * PDF attachment validation
  * windream DMS archiving support (placeholder)
  * Domain event publishing (EmailProcessedEvent)
  * Error handling and status tracking

Exception Improvements:
- Added ErrorCode property to DomainException
- Added ErrorCode overload to ValidationException
- Simplified AttachmentProcessingException to use base ErrorCode

Field Mappings Fixed:
- EmailProfile: ProcessId (not EmailProcessId), ValidationSql (not SenderFilter/SubjectFilter)
- EmailAccount: Username, EncryptedPassword, UseOAuth2, EncryptedClientSecret
- EmailHistory: SenderAddress, EmailDate, OriginalMessageId, EmailBodyText/Html
- EmailAttachment: OriginalFileName, SavedFileName, FilePath, FileSize
- Audit fields: AddedWhen/AddedWho, ChangedWhen/ChangedWho (not CreatedDate/By, ModifiedDate/By)

All commands follow Clean Architecture and use UnitOfWork pattern.
Build successful with 1 minor warning (dmsService marked for future implementation).
2026-07-08 15:51:51 +02:00
3778c0b338 feat(application): add repository and service interfaces
Repository Interfaces (Clean Architecture - Application Layer):
- IRepository<T>: Base repository interface with common CRUD operations
- IEmailAccountRepository: Email account operations (GetActive, GetByName, GetWithProfiles)
- IEmailProfileRepository: Profile operations (GetActive, GetDueForPolling, GetWithRelated)
- IEmailProcessRepository: Process operations (GetWithSteps, GetByType)
- IEmailHistoryRepository: History operations (pagination, duplicate detection, date range queries)
- IEmailOutboxRepository: Outbox operations (GetPending, GetForRetry, MarkAsSent/Failed)
- IUnitOfWork: Transaction management and repository aggregation

Service Interfaces (Abstraction for Infrastructure):
- IEmailService: IMAP/SMTP operations with OAuth2 support (MailKit wrapper)
- IPdfProcessingService: PDF validation, embedded file extraction, ZUGFeRD support
- IDmsService: windream DMS integration (archive, search, update index fields)
- IEncryptionService: Data protection for passwords and OAuth tokens
- IEmailQueue: Async email queue (in-memory Channel, future: RabbitMQ)

Dependencies:
- Added MimeKit 4.17.0 for email service interface definitions

All interfaces follow Clean Architecture principles:
- Interfaces in Application layer
- Implementations will be in Infrastructure layer
2026-07-08 10:39:16 +02:00
111d2bf264 fix(domain): use DateTime.Now instead of DateTime.UtcNow for legacy compatibility
CRITICAL FIX: Replace all DateTime.UtcNow with DateTime.Now throughout the application.

Reason: Legacy VB.NET system uses local server time, and database stores all
timestamps as local time. Using UTC breaks compatibility and causes incorrect
time comparisons.

Changes:
- EmailProcessedEvent: ProcessedDate now uses DateTime.Now
- EmailHistory.MarkAsProcessed(): ProcessedDate now uses DateTime.Now
- EmailHistory.MarkAsFailed(): ProcessedDate now uses DateTime.Now
- EmailProfile.UpdateLastPollTime(): LastPollTime now uses DateTime.Now
- EmailProfile.ShouldPoll(): Poll interval comparison now uses DateTime.Now

Documentation:
- Added critical note to agents.md about DateTime usage
- Includes examples and detailed explanation for future developers

This ensures all date/time operations remain compatible with legacy database.
2026-07-08 10:36:13 +02:00
43101a6e61 feat(application): add DTOs and application layer dependencies
DTOs:
- EmailProfileDto: Profile data transfer object
- EmailAccountDto: Email account data transfer object
- EmailHistoryDto: Email history data transfer object
- EmailAttachmentDto: Attachment data transfer object

Dependencies:
- MediatR 14.2.0 for CQRS (Commands/Queries)
- AutoMapper 12.0.1 for entity-DTO mapping
- FluentValidation 12.1.1 for input validation

This provides the foundation for the application layer implementation.
2026-07-07 18:59:31 +02:00
146b56ff85 build(domain): add MediatR dependency for domain events
- Add MediatR 12.2.0 package for event-driven architecture
- Enables domain events like EmailProcessedEvent
2026-07-07 18:59:22 +02:00
721603bb47 feat(domain): add domain services and events
- MessageIdGenerator: Generates unique message IDs using legacy-compatible SHA256 hash
  Algorithm matches VB.NET system for duplicate detection
- EmailProcessedEvent: MediatR domain event for email processing completion

Domain services encapsulate business logic that doesn't belong to entities.
2026-07-07 18:59:17 +02:00
c97073775b feat(domain): add all domain entities with legacy database mapping
Entities mapped to legacy database tables using [Table] and [Column] attributes:
- EmailAccount → TBDD_EMAIL_ACCOUNT (OAuth2 and password auth support)
- EmailProfile → TBEMLP_POLL_PROFILES (polling configuration)
- EmailProcess → TBEMLP_POLL_PROCESS (process definitions)
- ProcessStep → TBEMLP_POLL_STEPS (indexing steps)
- IndexingStep → TBEMLP_POLL_INDEXING_STEPS (DMS indexing fields)
- EmailHistory → TBEMLP_HISTORY (processed emails)
- EmailAttachment → TBEMLP_HISTORY_ATTACHMENT (email attachments)
- EmailOutbox → TBEMLP_EMAIL_OUT (outgoing email queue)

All entities follow Clean Architecture and DDD principles.
Database schema is read-only - no migrations will modify existing tables.
SNAKE_CASE columns mapped to PascalCase properties.
2026-07-07 18:59:10 +02:00
18bb07cd93 feat(domain): add domain exception hierarchy
- DomainException: Base exception for all domain errors
- ValidationException: Business rule validation failures
- AttachmentProcessingException: Attachment-specific processing errors

Exceptions maintain error code compatibility with legacy system.
2026-07-07 18:58:59 +02:00
f6946d812a feat(domain): add value objects for email domain
- MessageId: Unique message identifier with SHA256 hash
  Uses same algorithm as legacy system for duplicate detection compatibility
  Hash format: SHA256({originalMessageId}|{sender}|{date}|{subject})
- EmailAddress: Email address validation and parsing with name support

Value objects ensure immutability and value-based equality.
2026-07-07 18:58:54 +02:00
6098112bb4 feat(domain): add domain enumerations
- ErrorCode: All error codes from legacy system (10001-10010)
- ProcessType: Email process types (ProcessManager, AttachmentSniffer, ZugFeRDParser)
- AuthenticationType: Authentication methods (UsernamePassword, OAuth2)
- EmailStatus: Email processing status tracking
- AttachmentStatus: Attachment validation status

These enums maintain compatibility with the legacy VB.NET system.
2026-07-07 18:58:18 +02:00
05a36e8045 feat(domain): add common base classes and interfaces
- Add BaseEntity with audit fields (CreatedDate, CreatedBy, ModifiedDate, ModifiedBy)
- Add IAggregateRoot marker interface for DDD aggregate roots
- Add ValueObject base class with equality comparison by value

These classes provide the foundation for all domain entities and value objects.
2026-07-07 18:58:10 +02:00
915d01fc03 Add project references and fix encoding issue in tests
Added project references to establish dependencies between
the API, Application, Domain, and Infrastructure projects.
Updated `DigitalData.EmailProfiler.Tests.csproj` to include
references to all layers for testing purposes. Fixed a BOM
encoding issue in the test project file. Added xUnit usage
directive to ensure proper test framework integration.
2026-07-07 14:38:55 +02:00
690aee02dd Update solution and add new projects targeting .NET 8.0
Updated Visual Studio version in the solution file to 17.14.36717.8.
Added three new projects: Infrastructure, Domain, and Application,
all targeting .NET 8.0. Enabled implicit global usings and nullable
reference types in the new projects. Updated solution configuration
and nested the new projects under the `src` folder.
2026-07-07 13:31:56 +02:00
f3552dbdaa Add background service and update project configuration
Added a `Worker` class as a hosted background service to log
periodic messages. Updated `DigitalData.EmailProfiler.API.csproj`
to include `UserSecretsId` for secure development storage and
added `Microsoft.Extensions.Hosting` package. Replaced the
`Controllers` folder reference with `Properties`. Updated
`Program.cs` to register the `Worker` service, enable API
exploration, and retain Swagger configuration.
2026-07-07 13:25:58 +02:00
8a19a8a8bb init API 2026-07-07 13:18:24 +02:00