DigitalData.MessagingService/README.md

DigitalData.MessagingService

A .NET 8 messaging service for sending and receiving emails via SMTP, IMAP, POP3 and OAuth2, with RabbitMQ-based async delivery.

---

Email Account Configuration

Each account is configured under EmailAccounts.Accounts in appsettings.Secrets.json.

Different providers require different configuration fields. See provider-specific sections below.

Common fields (all providers)

{
  "Id": 1,
  "Username": "user@example.com",
  "Password": "your_password",
  "SmtpServer": "smtp.example.com",
  "SmtpPort": 465,
  "SmtpUseSsl": true,
  "UseOAuth2": false,
  "ImapServer": "imap.example.com",
  "ImapPort": 993,
  "ImapUseSsl": true,
  "Pop3Server": "pop.example.com",
  "Pop3Port": 995,
  "Pop3UseSsl": true,
  "IncomingProtocol": 1
}

Password is always retained. When UseOAuth2 = true, SMTP/IMAP/POP3 connections use OAuth2 tokens instead of the password. When UseOAuth2 = false, the password is used directly. The IncomingProtocol field independently controls which protocol is used for receiving emails.

IncomingProtocol values

Value	Meaning
0	None — send-only account, skipped by sync worker
1	IMAP with username/password
2	POP3 with username/password
3	IMAP with OAuth2
4	POP3 with OAuth2

OAuth2Provider values

Value	Meaning
0	None
1	Microsoft (Azure AD / Microsoft 365)
2	Google (Gmail / Google Workspace)

---

Provider-specific OAuth2 Configuration

Google (Gmail / Google Workspace)

Google uses user-delegated OAuth2 (authorization code flow). A one-time interactive authorization is required to obtain a refresh token. The refresh token is then stored in the database and reused automatically for all subsequent operations.

The refresh token survives application restarts. It will not be overwritten by the seed process unless OAuth2RefreshToken is explicitly set to a non-empty value in appsettings.Secrets.json.

Required fields

{
  "UseOAuth2": true,
  "OAuth2ClientId": "YOUR_CLIENT_ID.apps.googleusercontent.com",
  "OAuth2ClientSecret": "GOCSPX-YOUR_CLIENT_SECRET",
  "OAuth2RefreshToken": "",
  "OAuth2TenantId": "",
  "OAuth2Provider": 2
}

Google Cloud Console setup (one-time)

1. Go to console.cloud.google.com and create or select a project.
2. Enable the Gmail API under APIs & Services ? Library.
3. Go to APIs & Services ? Credentials ? + Create Credentials ? OAuth 2.0 Client ID:
   • Application type: Web application
   • Authorized redirect URIs: add https://YOUR_HOST/api/oauth2/google/callback (e.g. https://localhost:7261/api/oauth2/google/callback for local development)
4. Go to APIs & Services ? OAuth consent screen:
   • Add the Gmail account under Test users (required while app is in Testing mode).

Obtaining the refresh token via the built-in authorization endpoint

The application provides a built-in OAuth2 flow — no external tools needed.

1. Open a browser and navigate to:

   GET /api/oauth2/google/authorize/{accountId}
   

   Example: https://localhost:7261/api/oauth2/google/authorize/3

2. You will be redirected to Google's consent screen. If you see "Google hasn't verified this app", click Continue — this is expected while the app is in Testing mode.

3. Sign in with the Gmail account and grant access.

4. Google redirects back to /api/oauth2/google/callback automatically. The application exchanges the authorization code for a refresh token and saves it to the database.

5. A success response is returned:

   { "success": true, "username": "user@gmail.com", "message": "..." }
   

6. The sync worker and all IMAP/SMTP operations will now work automatically.

?? The refresh token must be re-obtained if invalid_grant is returned. This happens if the token is unused for 6 months or if the user revokes access.

---

Microsoft 365 / Exchange Online (Azure AD)

Microsoft uses application-level OAuth2 (client credentials flow — no user interaction required). Tokens are acquired automatically using the client ID, secret and tenant ID. No authorization endpoint needs to be visited.

Required fields

{
  "UseOAuth2": true,
  "OAuth2ClientId": "YOUR_APP_CLIENT_ID",
  "OAuth2ClientSecret": "YOUR_APP_CLIENT_SECRET_VALUE",
  "OAuth2TenantId": "yourorg.onmicrosoft.com",
  "OAuth2Provider": 1
}

Azure Portal setup (one-time)

1. Go to portal.azure.com ? Azure Active Directory ? App registrations ? + New registration.
2. Go to Certificates & secrets ? + New client secret ? copy the Value (not the ID).
   • Set this as OAuth2ClientSecret.
3. Go to API permissions ? + Add a permission ? APIs my organization uses ? Office 365 Exchange Online:
   • Add Application permissions: IMAP.AccessAsApp, SMTP.SendAsApp, POP.AccessAsApp
   • Click Grant admin consent
4. In Exchange Online PowerShell, register the service principal for the mailbox:

   New-ServicePrincipal -AppId <ClientId> -ServiceId <ObjectId> -DisplayName "MessagingService"
   Add-MailboxPermission -Identity "user@yourorg.onmicrosoft.com" -User <ObjectId> -AccessRights FullAccess
   

5. Set OAuth2TenantId to the full domain (e.g. yourorg.onmicrosoft.com) or tenant GUID.

?? OAuth2ClientSecret must be the Value shown at secret creation time, not the Secret ID (GUID). The value is only visible once — if lost, create a new secret.

?? OAuth2TenantId must be a full domain (yourorg.onmicrosoft.com), a tenant GUID, or common. Short names like yourorg are not valid and will cause AADSTS900023.

No browser-based authorization is required for Microsoft — the application acquires tokens automatically on first use and caches them in memory until 5 minutes before expiry.

---

Email Account Configuration

Each account is configured under EmailAccounts.Accounts in appsettings.Secrets.json.

Different providers require different configuration fields. See provider-specific sections below.

Common fields (all providers)

{
  "Id": 1,
  "Username": "user@example.com",
  "Password": "your_password",
  "SmtpServer": "smtp.example.com",
  "SmtpPort": 465,
  "SmtpUseSsl": true,
  "UseOAuth2": false,
  "ImapServer": "imap.example.com",
  "ImapPort": 993,
  "ImapUseSsl": true,
  "Pop3Server": "pop.example.com",
  "Pop3Port": 995,
  "Pop3UseSsl": true,
  "IncomingProtocol": 1
}

Password is always retained. When UseOAuth2 = true, SMTP/IMAP/POP3 connections use OAuth2 tokens instead of the password. When UseOAuth2 = false, the password is used directly. The IncomingProtocol field independently controls which protocol is used for receiving emails.

IncomingProtocol values

Value	Meaning
0	None — send-only account, skipped by sync worker
1	IMAP with username/password
2	POP3 with username/password
3	IMAP with OAuth2
4	POP3 with OAuth2

OAuth2Provider values

Value	Meaning
0	None
1	Microsoft (Azure AD / Microsoft 365)
2	Google (Gmail / Google Workspace)

---

Provider-specific OAuth2 Configuration

Google (Gmail / Google Workspace)

Google uses user-delegated OAuth2 (not client credentials). A one-time authorization flow is required to obtain a refresh token.

Required fields

{
  "UseOAuth2": true,
  "OAuth2ClientId": "YOUR_CLIENT_ID.apps.googleusercontent.com",
  "OAuth2ClientSecret": "GOCSPX-YOUR_CLIENT_SECRET",
  "OAuth2RefreshToken": "1//04YOUR_REFRESH_TOKEN",
  "OAuth2TenantId": "",
  "OAuth2Provider": 2
}

Setup — obtaining the refresh token (one-time)

1. Go to console.cloud.google.com and create or select a project.
2. Enable the Gmail API under APIs & Services ? Library.
3. Go to APIs & Services ? Credentials ? + Create Credentials ? OAuth 2.0 Client ID.
   • Application type: Web application
   • Authorized redirect URIs: https://developers.google.com/oauthplayground
4. Go to APIs & Services ? OAuth consent screen:
   • Add the Gmail account under Test users (required while app is in Testing mode).
5. Go to developers.google.com/oauthplayground:
   • Click ?? Settings ? enable "Use your own OAuth credentials"
   • Enter your Client ID and Client Secret (from step 3)
   • Close settings
6. In the scope input (Step 1), enter https://mail.google.com/ ? Authorize APIs
7. Sign in with the Gmail account ? grant access
8. Click Exchange authorization code for tokens (Step 2)
9. Copy the refresh_token value from the response
10. Set OAuth2RefreshToken in appsettings.Secrets.json

?? The refresh token must be obtained using your own client credentials in Playground settings. If obtained with Playground's default credentials, it will not work with your client secret.

?? Google refresh tokens expire if unused for 6 months, or if the user revokes access. The token must be re-obtained if invalid_grant is returned.

---

Microsoft 365 / Exchange Online (Azure AD)

Microsoft uses application-level OAuth2 (client credentials flow — no user interaction required).

Required fields

{
  "UseOAuth2": true,
  "OAuth2ClientId": "YOUR_APP_CLIENT_ID",
  "OAuth2ClientSecret": "YOUR_APP_CLIENT_SECRET_VALUE",
  "OAuth2TenantId": "yourorg.onmicrosoft.com",
  "OAuth2Provider": 1
}

Setup

1. Go to portal.azure.com ? Azure Active Directory ? App registrations ? + New registration.
2. Go to Certificates & secrets ? + New client secret ? copy the Value (not the ID).
   • Set this as OAuth2ClientSecret.
3. Go to API permissions ? + Add a permission ? APIs my organization uses ? Office 365 Exchange Online:
   • Add Application permissions: IMAP.AccessAsApp, SMTP.SendAsApp, POP.AccessAsApp
   • Click Grant admin consent
4. In Exchange Online PowerShell, register the service principal for the mailbox:

   New-ServicePrincipal -AppId <ClientId> -ServiceId <ObjectId> -DisplayName "MessagingService"
   Add-MailboxPermission -Identity "user@yourorg.onmicrosoft.com" -User <ObjectId> -AccessRights FullAccess
   

5. Set OAuth2TenantId to the full domain (e.g. yourorg.onmicrosoft.com) or tenant GUID.

?? OAuth2ClientSecret must be the Value shown at secret creation time, not the Secret ID (GUID). The value is only visible once — if lost, create a new secret.

?? OAuth2TenantId must be a full domain (yourorg.onmicrosoft.com), a tenant GUID, or common. Short names like yourorg are not valid and will cause AADSTS900023.