Compare commits

...

15 Commits

Author SHA1 Message Date
d01a0ceaa6 Refactor IMAP sync caching and improve thread safety
Replaced `IMemoryCache` with a thread-safe `ConcurrentDictionary`
for managing IMAP sync dates in `LimilabsImapEmailService`. Added
`GetLastImapSyncDate` and `SetLastImapSyncDate` methods to handle
cache operations. Updated the `IImapEmailService` interface to
include `GetLastImapSyncDate`. Simplified the `MarkAsSeenAsync`
method signature for better readability. Introduced a private
record type `ImapCacheKey` to encapsulate cache keys.
2026-08-14 08:40:56 +02:00
fd234618f7 chore: add legacy system analysis files for reference 2026-08-13 16:49:22 +02:00
f99bd8c399 fix(infrastructure): change IImapEmailService registration from Singleton to Scoped to support scoped DbContext dependency 2026-08-13 16:49:10 +02:00
c0bd391297 refactor(infrastructure): resolve IImapEmailService per-iteration from scoped DI in EmailSyncWorker; add structured logging and per-account error handling 2026-08-13 16:48:59 +02:00
cda70c8ced feat(infrastructure): replace DateFilter-based sync with IMemoryCache last-sync tracking in LimilabsImapEmailService; add CacheExtensions helper; return EmailSyncResult with processed/failed counts 2026-08-13 16:48:47 +02:00
578ecc7ba1 refactor(infrastructure): rename CreateAsync bulk overload to CreateRangeAsync and use AsNoTracking in ReceivedEmailRepository 2026-08-13 16:48:34 +02:00
6abe0e18f6 feat(application): add EmailSyncResult DTO, Folder/AccountId to ReceivedEmailDto, rename CreateRangeAsync in IRepository, update IImapEmailService signature, add AutoMapper profiles for ReceivedEmail and EmailAttachment 2026-08-13 16:48:22 +02:00
c9b7d99ecc feat(domain): add Folder property and change To/Cc to List<string> in ReceivedEmail 2026-08-13 16:48:06 +02:00
4964fa4344 Refactor IMAP service for repository integration
Replaced `FetchEmailsAsync` with `SyncEmailsAsync` in `IImapEmailService` to simplify email synchronization. Updated method signatures to use `DateFilter` and added default parameters for `folder` and `cancel`.

Refactored `LimilabsImapEmailService` to remove client-side filtering logic and delegate storage to `IRepository<ReceivedEmail>`. Simplified IMAP search criteria to focus on date-based filtering and added checks to skip already-synced emails.

Updated `EmailSyncWorker` to use the new `SyncEmailsAsync` method. Removed unused code, streamlined exception handling, and improved maintainability by reducing complexity and focusing on server-side filtering.
2026-08-13 14:18:43 +02:00
7207c5b4d9 Refactor email repository and query handling
Replaced `IMailRepository` with `IReceivedEmailRepository` to improve modularity and functionality. Updated `IReceivedEmailRepository` to make the `EmailAccount` parameter optional in the `FindAsync` method.

Added `ReceivedEmailRepository` with advanced filtering capabilities, including account, flags, text, UID, date, and recipient filters. Implemented deferred materialization for recipient filtering due to EF Core limitations.

Registered `IReceivedEmailRepository` in dependency injection. Updated `ReadEmailQueryHandler` to use the new repository. Improved query performance by applying filters conditionally.
2026-08-13 13:28:19 +02:00
7f04c4b09f Rename IMailRepository and add FindAsync method
Renamed the `IMailRepository` interface to `IReceivedEmailRepository`
to better reflect its purpose. Added a new `FindAsync` method to
the interface, which supports searching for received emails using
a `MailSearchFilter`, an `EmailAccount`, and an optional
`CancellationToken`. The method returns a `Task` resolving to
an `IEnumerable<ReceivedEmail>`.
2026-08-13 13:18:37 +02:00
43d7c393bb Refactor Repository to use protected DbSet field
Renamed `_dbSet` to `DbSet` and changed its accessibility from
`private` to `protected` to allow access in derived classes.

Updated all methods in the `Repository` class to use the new
`DbSet` field for querying, adding, updating, and removing
entities. This includes methods like `CreateAsync`,
`GetByIdAsync`, `FindAsync`, `UpsertAsync`, `UpdateAsync`,
and `DeleteAsync`.

Improved code consistency and readability by removing
redundant `_dbSet` references and standardizing on the
`DbSet` field.
2026-08-13 13:17:43 +02:00
e73bead2f2 Refactor email fetching to use IMailRepository
Introduced a new `IMailRepository` interface to abstract email-fetching logic, replacing the direct dependency on `IImapEmailService` in `ReadEmailQueryHandler`. Updated `ReadEmailQueryHandler` to use `IMailRepository` for querying emails and added `IMapper` for mapping entities to DTOs.

Modified the constructor of `ReadEmailQueryHandler` to inject `IMailRepository`, `IMapper`, and renamed `IRepository<EmailAccount>` to `EmailAccountRepo`. Replaced `IImapEmailService.FetchEmailsAsync` with `IMailRepository.FindAsync` in the handler's `Handle` method.

Wrapped all changes in `#if NET` preprocessor directives to ensure compatibility with specific build configurations. These changes improve modularity, testability, and separation of concerns.
2026-08-13 13:11:18 +02:00
606ff94a77 Rename FetchEmailsQuery to ReadEmailQuery
Renamed `FetchEmailsQuery` to `ReadEmailQuery` across the codebase, including its handler, validator, and usages in `EmailController`. Updated method signatures, dependencies, and XML documentation to reflect the new naming convention. Adjusted `ILogger` dependency in the handler to match the renamed class.
2026-08-13 12:09:44 +02:00
1c9af0e560 Remove caching from LimilabsImapEmailService
Simplified the `LimilabsImapEmailService` by removing the dependency on `IMemoryCache` and eliminating all caching logic. The constructor no longer accepts an `IMemoryCache` parameter, and the static `CacheKeyPrefix` field has been removed.

Replaced the caching mechanism with direct email fetching using `imap.GetMessageByUIDAsync`. Refactored the logic for processing attachments and visuals into `EmailAttachmentDto` objects, and streamlined the construction of `ReceivedEmailDto` to include metadata directly from the fetched email data.

These changes reduce complexity, improve maintainability, and ensure the service always retrieves the latest email data from the IMAP server.
2026-08-13 12:07:20 +02:00
17 changed files with 324 additions and 212 deletions

1
legacy Submodule

Submodule legacy added at e59b936181

View File

@@ -0,0 +1,5 @@
#if NET
namespace DigitalData.MessagingService.Application.Common.Dto;
public record EmailSyncResult(int ProcessedCount = 0, int FailedCount = 0);
#endif

View File

@@ -14,6 +14,15 @@ public sealed record ReceivedEmailDto
public long Uid { get; set; }
#endif
/// <summary>
/// ID of the email account this message belongs to.
/// </summary>
#if NET
public int AccountId { get; init; }
#else
public int AccountId { get; set; }
#endif
/// <summary>
/// Sender address (From header).
/// </summary>
@@ -94,4 +103,10 @@ public sealed record ReceivedEmailDto
#else
public bool IsSeen { get; set; }
#endif
#if NET
public required string Folder { get; init; }
#else
public string Folder { get; set; } = null!;
#endif
}

View File

@@ -1,6 +1,5 @@
#if NET
using DigitalData.MessagingService.Application.Common.Dto;
using DigitalData.MessagingService.Application.Common.Dto.MailSearch;
using DigitalData.MessagingService.Domain.Entities;
namespace DigitalData.MessagingService.Application.Common.Interfaces;
@@ -11,18 +10,16 @@ namespace DigitalData.MessagingService.Application.Common.Interfaces;
public interface IImapEmailService
{
/// <summary>
/// Fetches emails from the specified mailbox folder.
///
/// </summary>
/// <param name="account">Account whose IMAP settings will be used.</param>
/// <param name="filter">Filter to apply when fetching emails.</param>
/// When <see langword="true"/> (default), fetched messages are marked as <c>\Seen</c> on the server.
/// Set to <see langword="false"/> for a non-destructive read (uses <c>BODY.PEEK</c> internally).
/// </param>
/// <param name="cancellationToken">Cancellation token.</param>
Task<IEnumerable<ReceivedEmailDto>> FetchEmailsAsync(
/// <param name="account"></param>
/// <param name="folder"></param>
/// <param name="cancel"></param>
/// <returns></returns>
Task<EmailSyncResult> SyncEmailsAsync(
EmailAccount account,
MailSearchFilter filter,
CancellationToken cancellationToken = default);
string folder = "INBOX",
CancellationToken cancel = default);
/// <summary>
/// Marks a message as seen (read) on the server.
@@ -32,5 +29,13 @@ public interface IImapEmailService
long uid,
string folder = "INBOX",
CancellationToken cancellationToken = default);
/// <summary>
/// Gets the last IMAP sync date for the specified account and folder.
/// </summary>
/// <param name="accountId"></param>
/// <param name="folder"></param>
/// <returns></returns>
DateTime? GetLastImapSyncDate(int accountId, string folder = "INBOX");
}
#endif

View File

@@ -0,0 +1,11 @@
#if NET
using DigitalData.MessagingService.Application.Common.Dto.MailSearch;
using DigitalData.MessagingService.Domain.Entities;
namespace DigitalData.MessagingService.Application.Common.Interfaces.Repositories;
public interface IReceivedEmailRepository : IRepository<ReceivedEmail>
{
public Task<IEnumerable<ReceivedEmail>> FindAsync(MailSearchFilter mailSearchFilter, EmailAccount? accountQuery = null, CancellationToken cancellationToken = default);
}
#endif

View File

@@ -11,7 +11,7 @@ public interface IRepository<TEntity> where TEntity : class
// CREATE
Task<TEntity> CreateAsync<TDto>(TDto dto, CancellationToken cancellationToken = default);
Task<IEnumerable<TEntity>> CreateAsync<TDto>(IEnumerable<TDto> dtos, CancellationToken cancellationToken = default);
Task<IEnumerable<TEntity>> CreateRangeAsync<TDto>(IEnumerable<TDto> dtos, CancellationToken cancellationToken = default);
// READ
Task<TEntity?> GetByIdAsync(int id, CancellationToken cancellationToken = default);

View File

@@ -23,6 +23,19 @@ public class EmailMappingProfile : Profile
// EmailAccountDto -> EmailAccount
CreateMap<EmailAccount, EmailAccountDto>();
CreateMap<EmailAccountModificationDto, EmailAccount>();
// ReceivedEmailDto <-> ReceivedEmail
CreateMap<ReceivedEmailDto, ReceivedEmail>()
.ForMember(dest => dest.Id, opt => opt.Ignore())
.ForMember(dest => dest.Account, opt => opt.Ignore());
CreateMap<ReceivedEmail, ReceivedEmailDto>();
// EmailAttachmentDto <-> EmailAttachment
CreateMap<EmailAttachmentDto, EmailAttachment>()
.ForMember(dest => dest.Id, opt => opt.Ignore())
.ForMember(dest => dest.EmailId, opt => opt.Ignore())
.ForMember(dest => dest.Email, opt => opt.Ignore());
CreateMap<EmailAttachment, EmailAttachmentDto>();
}
}
#endif

View File

@@ -1,4 +1,5 @@
#if NET
using AutoMapper;
using DigitalData.MessagingService.Application.Common.Dto;
using DigitalData.MessagingService.Application.Common.Dto.MailSearch;
using DigitalData.MessagingService.Application.Common.Interfaces;
@@ -14,7 +15,7 @@ namespace DigitalData.MessagingService.Application.EmailReceiving.Queries;
/// <summary>
/// Query to fetch emails from an IMAP mailbox.
/// </summary>
public record FetchEmailsQuery : IRequest<IEnumerable<ReceivedEmailDto>>
public record ReadEmailQuery : IRequest<IEnumerable<ReceivedEmailDto>>
{
/// <summary>
/// Identifies the email account to use.
@@ -27,26 +28,24 @@ public record FetchEmailsQuery : IRequest<IEnumerable<ReceivedEmailDto>>
public MailSearchFilter Mail { get; init; } = new();
}
public class FetchEmailsQueryHandler(IImapEmailService ImapService, ILogger<FetchEmailsQueryHandler> Logger, IRepository<EmailAccount> Repo) : IRequestHandler<FetchEmailsQuery, IEnumerable<ReceivedEmailDto>>
public class ReadEmailQueryHandler(IMapper Mapper, ILogger<ReadEmailQueryHandler> Logger, IRepository<EmailAccount> EmailAccountRepo, IReceivedEmailRepository MailRepo) : IRequestHandler<ReadEmailQuery, IEnumerable<ReceivedEmailDto>>
{
public async Task<IEnumerable<ReceivedEmailDto>> Handle(FetchEmailsQuery request, CancellationToken cancellationToken)
public async Task<IEnumerable<ReceivedEmailDto>> Handle(ReadEmailQuery request, CancellationToken cancellationToken)
{
var accounts = await Repo.FindAsync(request.Account.Id is int id ? x => x.Id == id : x => x.Username == request.Account.Username, cancellationToken: cancellationToken);
var accounts = await EmailAccountRepo.FindAsync(request.Account.Id is int id ? x => x.Id == id : x => x.Username == request.Account.Username, cancellationToken: cancellationToken);
if (accounts.Count() > 1)
Logger.LogWarning("Multiple email accounts found for the given criteria ({Criteria}). Returning the first one.", request.Account.Id is not null ? $"Id: {request.Account.Id}" : $"Username: {request.Account.Username}");
EmailAccount account = accounts.FirstOrDefault()
var account = accounts.FirstOrDefault()
?? throw new NotFoundException($"No email account found for the given criteria (Id: {request.Account.Id}, Username: {request.Account.Username}).");
if (string.IsNullOrWhiteSpace(account.ImapServer))
throw new BadRequestException(
$"IMAP is not configured for account '{account.Username}' (Id: {account.Id}). Set ImapServer in EmailAccounts configuration.");
return await ImapService.FetchEmailsAsync(
account,
request.Mail,
cancellationToken);
var mails = await MailRepo.FindAsync(request.Mail, account, cancellationToken);
return Mapper.Map<IEnumerable<ReceivedEmailDto>>(mails);
}
}
#endif

View File

@@ -5,9 +5,9 @@ using FluentValidation;
namespace DigitalData.MessagingService.Application.EmailReceiving.Validators;
/// <summary>
/// Validates a <see cref="FetchEmailsQuery"/> before it is handled by <see cref="FetchEmailsQueryHandler"/>.
/// Validates a <see cref="ReadEmailQuery"/> before it is handled by <see cref="ReadEmailQueryHandler"/>.
/// </summary>
public class FetchEmailsQueryValidator : AbstractValidator<FetchEmailsQuery>
public class FetchEmailsQueryValidator : AbstractValidator<ReadEmailQuery>
{
public FetchEmailsQueryValidator()
{

View File

@@ -45,9 +45,9 @@ public sealed record ReceivedEmail
/// </summary>
[Column("TO", TypeName = "nvarchar(max)")]
#if NET
public IEnumerable<string> To { get; init; } = [];
public List<string> To { get; init; } = [];
#else
public IEnumerable<string> To { get; set; } = [];
public List<string> To { get; set; } = [];
#endif
/// <summary>
@@ -55,9 +55,9 @@ public sealed record ReceivedEmail
/// </summary>
[Column("CC", TypeName = "nvarchar(max)")]
#if NET
public IEnumerable<string> Cc { get; init; } = [];
public List<string> Cc { get; init; } = [];
#else
public IEnumerable<string> Cc { get; set; } = [];
public List<string> Cc { get; set; } = [];
#endif
/// <summary>
@@ -133,4 +133,10 @@ public sealed record ReceivedEmail
#else
public EmailAccount? Account { get; set; }
#endif
#if NET
public required string Folder { get; init; }
#else
public string Folder { get; set; } = null!;
#endif
}

View File

@@ -34,7 +34,7 @@ public static class DependencyInjection
// Email Service - IMAP inbound (Limilabs Mail.dll)
// Fresh connection per call — stateless and thread-safe.
services.AddSingleton<IImapEmailService, LimilabsImapEmailService>();
services.AddScoped<IImapEmailService, LimilabsImapEmailService>();
// PDF Processing Service (using DevExpress.Pdf)
services.AddScoped<IPdfProcessingService, DevExpressPdfProcessingService>();
@@ -66,6 +66,8 @@ public static class DependencyInjection
services.AddScoped(typeof(IRepository<>), typeof(Repository<>));
services.AddScoped<IReceivedEmailRepository, ReceivedEmailRepository>();
// AutoMapper - Register entity self-mappings (T -> T) for generic repository
services.AddAutoMapper(config => config.AddMaps(typeof(EntitySelfMappingProfile).Assembly));

View File

@@ -0,0 +1,83 @@
using AutoMapper;
using DigitalData.MessagingService.Application.Common.Dto.MailSearch;
using DigitalData.MessagingService.Application.Common.Interfaces.Repositories;
using DigitalData.MessagingService.Domain.Entities;
using DigitalData.MessagingService.Infrastructure.Persistence;
using Microsoft.EntityFrameworkCore;
namespace DigitalData.MessagingService.Infrastructure.Repositories;
public class ReceivedEmailRepository(MessagingServiceDbContext Context, IMapper Mapper) : Repository<ReceivedEmail>(Context, Mapper), IReceivedEmailRepository
{
public async Task<IEnumerable<ReceivedEmail>> FindAsync(MailSearchFilter mailSearchFilter, EmailAccount? accountQuery = null, CancellationToken cancellationToken = default)
{
var query = DbSet.AsNoTracking();
// ── Account filter ─────────────────────────────────────────────────────
if (accountQuery is not null)
query = query.Where(x => x.AccountId == accountQuery.Id);
// ── Flag filters ───────────────────────────────────────────────────────
if (mailSearchFilter.UnseenOnly)
query = query.Where(x => !x.IsSeen);
// ── Text filters ───────────────────────────────────────────────────────
if (!string.IsNullOrWhiteSpace(mailSearchFilter.SubjectContains))
query = query.Where(x => x.Subject.Contains(mailSearchFilter.SubjectContains));
if (!string.IsNullOrWhiteSpace(mailSearchFilter.SenderContains))
query = query.Where(x => x.From.Contains(mailSearchFilter.SenderContains));
if (!string.IsNullOrWhiteSpace(mailSearchFilter.BodyContains))
query = query.Where(x => x.TextBody.Contains(mailSearchFilter.BodyContains)
|| x.HtmlBody.Contains(mailSearchFilter.BodyContains));
// ── UID filter ─────────────────────────────────────────────────────────
if (mailSearchFilter.Uid is { } uid)
{
if (uid.Absolute.HasValue)
query = query.Where(x => x.Uid == uid.Absolute.Value);
else
{
if (uid.Min.HasValue)
query = query.Where(x => x.Uid >= uid.Min.Value);
if (uid.Max.HasValue)
query = query.Where(x => x.Uid <= uid.Max.Value);
}
}
// ── Date filter ────────────────────────────────────────────────────────
if (mailSearchFilter.Date is { } date)
{
if (date.After.HasValue)
query = query.Where(x => x.Date >= date.After.Value);
if (date.Before.HasValue)
query = query.Where(x => x.Date <= date.Before.Value);
}
// ── Attachments ────────────────────────────────────────────────────────
if (mailSearchFilter.WithAttachments)
query = query.Include(x => x.Attachments);
// ── Sort ───────────────────────────────────────────────────────────────
query = mailSearchFilter.SortOrder == MailSortOrder.OldestFirst
? query.OrderBy(x => x.Date)
: query.OrderByDescending(x => x.Date);
// ── Limit ──────────────────────────────────────────────────────────────
if (mailSearchFilter.MaxCount.HasValue)
query = query.Take(mailSearchFilter.MaxCount.Value);
// ── RecipientContains: To/Cc are IEnumerable<string> (nvarchar(max)) ──
// EF Core cannot translate collection predicates on these columns to SQL.
// Materialization is deferred until after other DB-side filters narrow the set.
var results = await query.ToListAsync(cancellationToken);
if (!string.IsNullOrWhiteSpace(mailSearchFilter.RecipientContains))
results = [.. results
.Where(x => x.To.Any(t => t.Contains(mailSearchFilter.RecipientContains, StringComparison.OrdinalIgnoreCase))
|| x.Cc.Any(c => c.Contains(mailSearchFilter.RecipientContains, StringComparison.OrdinalIgnoreCase)))];
return results;
}
}

View File

@@ -13,22 +13,22 @@ namespace DigitalData.MessagingService.Infrastructure.Repositories;
/// </summary>
public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapper) : IRepository<TEntity> where TEntity : class
{
private readonly DbSet<TEntity> _dbSet = Context.Set<TEntity>();
protected readonly DbSet<TEntity> DbSet = Context.Set<TEntity>();
// --- CREATE ---
public async Task<TEntity> CreateAsync<TDto>(TDto dto, CancellationToken cancellationToken = default)
{
var entity = Mapper.Map<TEntity>(dto);
await _dbSet.AddAsync(entity, cancellationToken);
await DbSet.AddAsync(entity, cancellationToken);
await Context.SaveChangesAsync(cancellationToken);
return entity;
}
public async Task<IEnumerable<TEntity>> CreateAsync<TDto>(IEnumerable<TDto> dtos, CancellationToken cancellationToken = default)
public async Task<IEnumerable<TEntity>> CreateRangeAsync<TDto>(IEnumerable<TDto> dtos, CancellationToken cancellationToken = default)
{
var entities = Mapper.Map<IEnumerable<TEntity>>(dtos);
await _dbSet.AddRangeAsync(entities, cancellationToken);
await DbSet.AddRangeAsync(entities, cancellationToken);
await Context.SaveChangesAsync(cancellationToken);
return entities;
}
@@ -37,12 +37,12 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
public async Task<TEntity?> GetByIdAsync(int id, CancellationToken cancellationToken = default)
{
return await _dbSet.FindAsync([id], cancellationToken);
return await DbSet.FindAsync([id], cancellationToken);
}
public async Task<IEnumerable<TEntity>> GetAllAsync(CancellationToken cancellationToken = default)
{
return await _dbSet.ToListAsync(cancellationToken);
return await DbSet.ToListAsync(cancellationToken);
}
public async Task<IEnumerable<TEntity>> FindAsync(
@@ -51,7 +51,7 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
int? take = null,
CancellationToken cancellationToken = default)
{
var query = _dbSet.Where(predicate);
var query = DbSet.Where(predicate);
if (skip.HasValue)
query = query.Skip(skip.Value);
@@ -66,14 +66,14 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
Expression<Func<TEntity, bool>> predicate,
CancellationToken cancellationToken = default)
{
return await _dbSet.FirstOrDefaultAsync(predicate, cancellationToken);
return await DbSet.FirstOrDefaultAsync(predicate, cancellationToken);
}
public async Task<TEntity?> FindSingleAsync(
Expression<Func<TEntity, bool>> predicate,
CancellationToken cancellationToken = default)
{
return await _dbSet.SingleOrDefaultAsync(predicate, cancellationToken);
return await DbSet.SingleOrDefaultAsync(predicate, cancellationToken);
}
public async Task<int> CountAsync(
@@ -81,15 +81,15 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
CancellationToken cancellationToken = default)
{
return predicate == null
? await _dbSet.CountAsync(cancellationToken)
: await _dbSet.CountAsync(predicate, cancellationToken);
? await DbSet.CountAsync(cancellationToken)
: await DbSet.CountAsync(predicate, cancellationToken);
}
public async Task<bool> AnyAsync(
Expression<Func<TEntity, bool>> predicate,
CancellationToken cancellationToken = default)
{
return await _dbSet.AnyAsync(predicate, cancellationToken);
return await DbSet.AnyAsync(predicate, cancellationToken);
}
// --- UPSERT ---
@@ -105,12 +105,12 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
TDto dto,
CancellationToken cancellationToken = default)
{
var entity = await _dbSet.FirstOrDefaultAsync(predicate, cancellationToken);
var entity = await DbSet.FirstOrDefaultAsync(predicate, cancellationToken);
if (entity is null)
{
entity = Mapper.Map<TEntity>(dto);
await _dbSet.AddAsync(entity, cancellationToken);
await DbSet.AddAsync(entity, cancellationToken);
await Context.SaveChangesAsync(cancellationToken);
return (entity, true);
}
@@ -130,12 +130,12 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
TDto dto,
CancellationToken cancellationToken = default)
{
var entity = await _dbSet.SingleOrDefaultAsync(predicate, cancellationToken);
var entity = await DbSet.SingleOrDefaultAsync(predicate, cancellationToken);
if (entity is null)
{
entity = Mapper.Map<TEntity>(dto);
await _dbSet.AddAsync(entity, cancellationToken);
await DbSet.AddAsync(entity, cancellationToken);
await Context.SaveChangesAsync(cancellationToken);
return (entity, true);
}
@@ -157,7 +157,7 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
TDto dto,
CancellationToken cancellationToken = default)
{
var entity = await _dbSet.SingleOrDefaultAsync(predicate, cancellationToken)
var entity = await DbSet.SingleOrDefaultAsync(predicate, cancellationToken)
?? throw new NotFoundException($"No {typeof(TEntity).Name} found matching the predicate.");
Mapper.Map(dto, entity);
await Context.SaveChangesAsync(cancellationToken);
@@ -173,7 +173,7 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
TDto dto,
CancellationToken cancellationToken = default)
{
var entities = await _dbSet.Where(predicate).ToListAsync(cancellationToken);
var entities = await DbSet.Where(predicate).ToListAsync(cancellationToken);
entities.ForEach(entity => Mapper.Map(dto, entity));
await Context.SaveChangesAsync(cancellationToken);
return entities.Count;
@@ -190,9 +190,9 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
Expression<Func<TEntity, bool>> predicate,
CancellationToken cancellationToken = default)
{
var entity = await _dbSet.SingleOrDefaultAsync(predicate, cancellationToken)
var entity = await DbSet.SingleOrDefaultAsync(predicate, cancellationToken)
?? throw new NotFoundException($"No {typeof(TEntity).Name} found matching the predicate.");
_dbSet.Remove(entity);
DbSet.Remove(entity);
await Context.SaveChangesAsync(cancellationToken);
}
@@ -205,8 +205,8 @@ public class Repository<TEntity>(MessagingServiceDbContext Context, IMapper Mapp
Expression<Func<TEntity, bool>> predicate,
CancellationToken cancellationToken = default)
{
var entities = await _dbSet.Where(predicate).ToListAsync(cancellationToken);
_dbSet.RemoveRange(entities);
var entities = await DbSet.Where(predicate).ToListAsync(cancellationToken);
DbSet.RemoveRange(entities);
await Context.SaveChangesAsync(cancellationToken);
return entities.Count;
}

View File

@@ -1,28 +1,22 @@
using DigitalData.MessagingService.Application.Common.Dto.MailSearch;
using DigitalData.MessagingService.Application.Common.Interfaces;
using DigitalData.MessagingService.Application.Common.Interfaces.Repositories;
using DigitalData.MessagingService.Application.Common.Options;
using DigitalData.MessagingService.Domain.Entities;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace DigitalData.MessagingService.Infrastructure.Services.Background;
public class EmailSyncWorker(IImapEmailService imapService, IOptions<EmailAccountsOptions> Options, IServiceProvider Provider) : BackgroundService
public class EmailSyncWorker(IOptions<EmailAccountsOptions> Options, IServiceProvider Provider, ILogger<EmailSyncWorker> Logger) : BackgroundService
{
private DateFilter? _dateFilter = null;
private readonly string DefaultFolder = "INBOX";
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await UpsertSeedEmailAccount(stoppingToken);
if (imapService is not LimilabsImapEmailService limapService)
{
await Task.Delay(Timeout.Infinite, stoppingToken).ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);
return;
}
var interval = TimeSpan.FromSeconds(Options.Value.SyncIntervalSeconds);
while (!stoppingToken.IsCancellationRequested)
@@ -31,23 +25,23 @@ public class EmailSyncWorker(IImapEmailService imapService, IOptions<EmailAccoun
var emailAccountRepo = scope.ServiceProvider.GetRequiredService<IRepository<EmailAccount>>();
var imapService = scope.ServiceProvider.GetRequiredService<IImapEmailService>();
foreach (var account in await emailAccountRepo.GetAllAsync(stoppingToken))
if (account.ImapServer is not null)
{
// init or update last date filter
_dateFilter = _dateFilter is null
? new DateFilter
{
After = null,
Before = DateTime.UtcNow
}
: new DateFilter
{
After = _dateFilter.Before,
Before = DateTime.UtcNow
};
Logger.LogDebug("Email synchronization has started for account {username} in folder {folder}.", account.Username, DefaultFolder);
await limapService.FetchEmailsAsync(account, new MailSearchFilter { Date = _dateFilter }, stoppingToken);
try
{
var res = await imapService.SyncEmailsAsync(account, DefaultFolder, stoppingToken);
Logger.LogDebug("Email synchronization has completed for account {username} in folder {folder}. Processed: {processedCount}, Failed: {failedCount}", account.Username, DefaultFolder, res.ProcessedCount, res.FailedCount);
}
catch(Exception ex)
{
// Log the exception or handle it as needed
Logger.LogError(ex, "Error syncing emails for account {username}", account.Username);
}
}
await Task.Delay(interval, stoppingToken).ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);

View File

@@ -0,0 +1,24 @@
using Microsoft.Extensions.Caching.Memory;
namespace DigitalData.MessagingService.Infrastructure.Services.Extensions;
public static class CacheExtensions
{
private readonly static string ImapCacheKeyPrefix = Guid.NewGuid().ToString();
private static string CreateImapLastSyncDateCacheKey(int accountId, string folder)
{
return $"{ImapCacheKeyPrefix}_{accountId}_{folder}_LastImapSyncDate";
}
public static DateTime? GetLastImapSyncDate(this IMemoryCache cache, int accountId, string folder)
{
return cache.Get<DateTime?>(CreateImapLastSyncDateCacheKey(accountId, folder));
}
public static void SetLastImapSyncDate(this IMemoryCache cache, int accountId, string folder, DateTime date)
{
var key = CreateImapLastSyncDateCacheKey(accountId, folder);
cache.Set(key, date);
}
}

View File

@@ -1,13 +1,12 @@
using DigitalData.MessagingService.Application.Common.Dto;
using DigitalData.MessagingService.Application.Common.Dto.MailSearch;
using DigitalData.MessagingService.Application.Common.Interfaces;
using DigitalData.MessagingService.Application.Common.Interfaces.Repositories;
using DigitalData.MessagingService.Domain.Entities;
using DigitalData.MessagingService.Domain.Exceptions;
using DigitalData.MessagingService.Infrastructure.Services.Extensions;
using Limilabs.Client.IMAP;
using Limilabs.Mail;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using System.Collections.Concurrent;
using System.Text;
namespace DigitalData.MessagingService.Infrastructure.Services;
@@ -16,7 +15,7 @@ namespace DigitalData.MessagingService.Infrastructure.Services;
/// IMAP email service using Limilabs Mail.dll.
/// Opens a fresh connection per call — stateless and thread-safe.
/// </summary>
public class LimilabsImapEmailService(ILogger<LimilabsImapEmailService> Logger, IMemoryCache Cache) : IImapEmailService
public class LimilabsImapEmailService(ILogger<LimilabsImapEmailService> Logger, IRepository<ReceivedEmail> Repository) : IImapEmailService
{
private static readonly string CacheKeyPrefix = Guid.NewGuid().ToString();
@@ -26,51 +25,44 @@ public class LimilabsImapEmailService(ILogger<LimilabsImapEmailService> Logger,
}
// Public API
public async Task<IEnumerable<ReceivedEmailDto>> FetchEmailsAsync(
EmailAccount account,
MailSearchFilter filter,
CancellationToken cancel = default)
public async Task<EmailSyncResult> SyncEmailsAsync(EmailAccount account, string folder = "INBOX", CancellationToken cancel = default)
{
using var imap = await OpenAsync(account, filter.Folder, cancel);
using var imap = await OpenAsync(account, folder, cancel);
try
{
#region Find UIDs
// Server-side: only date range; all other filters are applied in-process after cache retrieval
List<ICriterion> criterions = [];
if (filter.Date is DateFilter dateF)
{
if (dateF.After is DateTime after)
criterions.Add(Expression.SentSince(after.Date));
var since = GetLastImapSyncDate(account.Id, folder);
// IMAP BEFORE is exclusive, so add one day to make the bound inclusive
if (dateF.Before is DateTime before)
criterions.Add(Expression.SentBefore(before.Date.AddDays(1)));
}
if (since is not null && since != default)
criterions.Add(Expression.SentSince(since.Value));
var searchExpression = criterions.Count > 0 ? Expression.And([.. criterions]) : Expression.All();
List<long> uids = [.. await imap.SearchAsync(searchExpression, cancel)];
if (filter.SortOrder == MailSortOrder.NewestFirst)
uids.Reverse();
var operationStartTime = DateTime.UtcNow;
List<long> uids = [.. await imap.SearchAsync(searchExpression, cancel)];
#endregion
if (uids.Count == 0)
return [];
return new EmailSyncResult();
var results = new List<ReceivedEmailDto>(uids.Count);
var emails = new List<ReceivedEmailDto>(uids.Count);
var failedCount = 0;
foreach (var uid in uids)
{
if (await Repository.AnyAsync(x => x.Uid == uid, cancel))
continue;
cancel.ThrowIfCancellationRequested();
try
{
#region Read email
var email = await Cache.GetOrCreateAsync(
CacheKeyPrefix + uid,
async entry =>
{
var eml = await imap.GetMessageByUIDAsync(uid, cancel);
var mail = new MailBuilder().CreateFromEml(eml);
var flags = await imap.GetFlagsByUIDAsync(uid, cancel);
@@ -101,9 +93,10 @@ public class LimilabsImapEmailService(ILogger<LimilabsImapEmailService> Logger,
});
}
return new ReceivedEmailDto
var email = new ReceivedEmailDto
{
Uid = uid,
AccountId = account.Id,
From = mail.From.FirstOrDefault()?.Address ?? string.Empty,
To = [.. mail.To.SelectMany(m => m.GetMailboxes()).Select(mb => mb.Address)],
Cc = [.. mail.Cc.SelectMany(m => m.GetMailboxes()).Select(mb => mb.Address)],
@@ -113,86 +106,37 @@ public class LimilabsImapEmailService(ILogger<LimilabsImapEmailService> Logger,
Date = mail.Date ?? DateTime.MinValue,
IsSeen = flags.Contains(Flag.Seen),
Attachments = attachments,
Folder = folder
};
});
#endregion Read email
if (email is null)
continue;
if (filter.UnseenOnly && email.IsSeen)
continue;
if (filter.SubjectContains is string subject &&
!email.Subject.Contains(subject, StringComparison.OrdinalIgnoreCase))
continue;
if (filter.SenderContains is string sender &&
!email.From.Contains(sender, StringComparison.OrdinalIgnoreCase))
continue;
if (filter.RecipientContains is string recipient &&
!email.To.Any(t => t.Contains(recipient, StringComparison.OrdinalIgnoreCase)) &&
!email.Cc.Any(c => c.Contains(recipient, StringComparison.OrdinalIgnoreCase)))
continue;
if (filter.BodyContains is string body &&
!email.TextBody.Contains(body, StringComparison.OrdinalIgnoreCase) &&
!email.HtmlBody.Contains(body, StringComparison.OrdinalIgnoreCase))
continue;
if (filter.Uid is UidFilter uidF)
{
if (uidF.Absolute is long exactUid && email.Uid != exactUid)
continue;
if (uidF.Min is long min && email.Uid < min)
continue;
if (uidF.Max is long max && email.Uid > max)
continue;
}
if (filter.WithAttachments)
results.Add(email);
else
results.Add(email with { Attachments = [] });
emails.Add(email);
SetLastImapSyncDate(account.Id, folder, operationStartTime);
}
catch (Exception ex)
{
failedCount += 1;
Logger.LogWarning(ex,
"Failed to fetch IMAP message UID={Uid} from folder {Folder}. Skipping.",
uid, filter.Folder);
uid, folder);
}
}
await imap.CloseAsync(cancel);
if (filter.MaxCount is int maxCount && maxCount > 0 && results.Count > maxCount)
return results.Take(maxCount);
await Repository.CreateRangeAsync(emails, cancel);
return results;
return new EmailSyncResult(ProcessedCount: emails.Count, FailedCount: failedCount);
}
catch (Limilabs.Client.ServerException ex)
catch
{
await imap.CloseSafelyAsync();
throw new AuthenticationFailedException(
$"IMAP authentication failed for account '{account.Username}'.", ex);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
await imap.CloseSafelyAsync();
throw new InvalidOperationException(
$"Failed to fetch emails from IMAP server '{account.ImapServer}'.", ex);
throw;
}
}
public async Task MarkAsSeenAsync(
EmailAccount account,
long uid,
string folder = "INBOX",
CancellationToken cancel = default)
public async Task MarkAsSeenAsync(EmailAccount account, long uid, string folder = "INBOX", CancellationToken cancel = default)
{
using var imap = await OpenAsync(account, folder, cancel);
try
@@ -200,21 +144,14 @@ public class LimilabsImapEmailService(ILogger<LimilabsImapEmailService> Logger,
await imap.MarkMessageSeenByUIDAsync(uid, cancel);
await imap.CloseAsync(cancel);
}
catch (Limilabs.Client.ServerException ex)
catch
{
await imap.CloseSafelyAsync();
throw new AuthenticationFailedException(
$"IMAP authentication failed for account '{account.Username}'.", ex);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
await imap.CloseSafelyAsync();
throw new InvalidOperationException(
$"Failed to mark message UID={uid} as seen on '{account.ImapServer}'.", ex);
throw;
}
}
private static async Task<Imap> OpenAsync(EmailAccount account, string folder, CancellationToken cancel)
private static async Task<Imap> OpenAsync(EmailAccount account, string folder = "INBOX", CancellationToken cancel = default)
{
var imap = new Imap();
@@ -232,4 +169,21 @@ public class LimilabsImapEmailService(ILogger<LimilabsImapEmailService> Logger,
return imap;
}
#region IMAP Last Sync Date Cache
private readonly ConcurrentDictionary<ImapCacheKey, DateTime> _cache = new();
private record ImapCacheKey(int AccountId, string Folder);
public DateTime? GetLastImapSyncDate(int accountId, string folder = "INBOX")
{
return _cache.GetValueOrDefault(new ImapCacheKey(accountId, folder));
}
private void SetLastImapSyncDate(int accountId, string folder, DateTime date)
{
var key = new ImapCacheKey(accountId, folder);
_cache[key] = date;
}
#endregion
}

View File

@@ -92,7 +92,7 @@ public class EmailController(IMediator mediator) : ControllerBase
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status400BadRequest)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
public async Task<IActionResult> FetchEmails([FromQuery] FetchEmailsQuery query, [FromQuery] OnlyFilter? only = null, CancellationToken cancellationToken = default)
public async Task<IActionResult> FetchEmails([FromQuery] ReadEmailQuery query, [FromQuery] OnlyFilter? only = null, CancellationToken cancellationToken = default)
{
var emails = await mediator.Send(query, cancellationToken);