Metadata Exchange (data seed)

Overview

The basic idea of exchanging metadata is to ensure the transfer of metadata from the application domain to the platform domain.

The application domain defines the content of the metadata, and the platform domain defines their format and uses the metadata for its activities.

At the same time, it is necessary to carry out an exchange so that there is no need for coordination on development at the same time. Therefore, if the update is to occur, it must be done without coordinating the development teams of the platform domain and the application domain.

Metadata exchange patterns

1. Active (Push) - The application domain is active and pushes them to the platform domain. This scenario is useful if the metadata is static and already known at the time the domain is deployed to the target environment. From the point of view of the principle of tenancy, these are metadata that are the same for all tenants.
2. Passive (Pull) - The application domain is passive, exposing the endpoint to which it exposes its metadata. Endpoint has an interface (routing, parameters) according to the regulations of the platform service. The platform service is configured which passive services to contact to obtain metadata. This scenario is useful if the dynamic metadata is dependent on data and other settings in the application domain.

Metadata format recommendations

• Format Versioning - Simply for recognizing what version of the metadata format is being sent. Thus, the platform service can respond flexibly to legacy formats.
• Content Versioning - In some cases, it is a good idea for the metadata to contain a version of the content to easily reduce the amount of content sent and quickly check if a new transfer and update is required.

Active exchange (Push)

Platform domain (service) implementation (server part)

• Metadata model contracts
• API Endpoint - Controller with authorization.
• Facade - To convert metadata model into platform domain entities and execute command to save metadata model into domain.
• ImportMetadataCommand and ImportMetadataCommandHandler - Create/update/delete domain entities for metadata.
• Connector - for easy importing a new definition of metadata from application domain.

Sample implementation of endpoint (RestAPI) to process metadata import

[ApiController]
[Route("api/v{version:apiVersion}/[controller]")]
[Consumes("application/json")]
[Produces("application/json")]
[ApiConventionType(typeof(DefaultApiConventions))]
public class ProcessController : AuthorizeControllerBase
{
    public ProcessController(
            IProcessFacade facade,
            ILogger<ProcessController> logger)
            : base(logger)
        {
            Facade = facade;
        }
    }
    
    protected IProcessFacade Facade { get; }

    /// <summary>
    /// Import metadata from application domain into platform domain
    /// </summary>
    /// <param name="taskModels"></param>
    /// <param name="ct"></param>
    /// <returns></returns>
    [HttpPost]
    [Route("Metadata/Import")]
    [ProducesResponseType(StatusCodes.Status200OK, Type = typeof(MetadataImportModelResult))]
    public async Task<IActionResult> MetadataImportAsync([FromBody] MetadataImportModel model, CancellationToken ct = default)
    {
        var result = await Facade.MetadataImportAsync(User, model, ct);
        return result != null ? (IActionResult)Ok(result) : NoContent();
    }
}

Sample implementation of Facade to process metadata import

public class ProcessFacade : IProcessFacade
{
    public ProcessFacade(
        IMapper mapper,
        IQueryProcessor queryProcessor,
        ICommandProcessor commandProcessor)
    {
        Mapper = mapper;
        QueryProcessor = queryProcessor;
        CommandProcessor = commandProcessor;
    }

    protected IMapper Mapper { get; }
    protected IQueryProcessor QueryProcessor { get; }
    protected ICommandProcessor CommandProcessor { get; }

    public async Task<MetadataImportModelResult> MetadataImportAsync(ClaimsPrincipal user, MetadataImportModel model, CancellationToken ct)
    {
        var command = new MetadataImportCommand(user, model);
        var commandResult = await CommandProcessor.ProcessAsync<MetadataImportCommand, MetadataImportModelResult>(command, ct);
        return FacadeHelper.ResolveCommandResultValue(commandResult);
    }
}

Sample implementation of CommandHandler to process metadata import

public class MetadataImportCommandHandler : ICommandHandler<MetadataImportCommand, MetadataImportModelResult>
{
    public MetadataImportCommandHandler(
        IMapper mapper,
        IMasterDbScope<IMetadataImportRepository> imports, 
        IMasterDbScope<IMetadataRepository> metadata,
        ILogger<MetadataImportCommandHandler> logger
    )
    {
        ImportRepository = imports.Instance;
        MetadataRepository = metadata.Instance;
        Mapper = mapper;
        Log = logger;
    }

    // Repository of domain entity for import (header). Contains content version for every application by ApplicationCode.
    protected IMetadataImportRepository ImportRepository { get; }

    // Repository of domain entity for metadata.
    protected IMetadataRepository MetadataRepository { get; }

    protected IMapper Mapper { get; }
    protected ILogger Log { get; }

    public async Task<ExecutionResult<MetadataImportModelResult>> HandleAsync(MetadataImportCommand command, CancellationToken ct = default)
    {
        // ..Merge existing metadata, delete missing metadata, create new metadata
    }
}

Platform domain (service) implementation (interface part)

Sample implementation of Connector (client)

public class PlatformClient : SecuredApiClientBase<PlatformClientOptions, IConnectorTokenProvider>, IPlatformClient
{
    protected override string ApiVersionPrefix => "api/v1";

    public PlatformClient (
        IOptions<PlatformClientOptions> options,
        ILogger<PlatformClient> logger,
        IConnectorTokenProvider tokenProvider)
        : base(options.Value, logger, tokenProvider)
    {
    }

    public PlatformClient (
        PlatformClientOptions options,
        ILogger<PlatformClient> logger,
        IConnectorTokenProvider tokenProvider)
        : base(options, logger, tokenProvider)
    {
    }

    public async Task MetadataImportAsync(string resourceDefinitionPath, Assembly assembly, CancellationToken ct)
    {
        var embeddedProvider = new EmbeddedFileProvider(assembly);
        string content = string.Empty;
        using (var reader = embeddedProvider.GetFileInfo(resourceDefinitionPath).CreateReadStream())
        using (var sr = new System.IO.StreamReader(reader))
        {
            content = await sr.ReadToEndAsync();
        }
        var model = Newtonsoft.Json.JsonConvert.DeserializeObject<MetadataModelImport>(content);
        await TaskImportAsync(model, ct);
    }

    public async Task MetadataImportAsync(MetadataModelImport model, CancellationToken ct)
    {
        var resource = $"{ApiVersionPrefix}/Process/Metadata/Import";
        Logger.LogInformation($"Push metadata into {CombineUri(resource)}");
        var request = (await AddAuthentication(Client.PostAsync(resource, model), ct))
             .WithCancellationToken(ct)
             .WithOptions(false, false)
             .AddDefaultRetryPolicy();

        await request.As<MetadataImportModelResult>();
    }
}

Application domain (service) implementation (client part)

• Use nuget Connector/Contracts from platform domain.
• MetadataImporterService - Background service to allow import new metadata content at service start.

Sample implementation of MetadataImporter background service

public class MetadataImporterService : BackgroundService
{
    public MetadataImporterService(IServiceProvider serviceProvider)
    {
        ServiceProvider = serviceProvider;
    }

    public IServiceProvider ServiceProvider { get; }

    // Connector platformServiceClient load embedded resource .json from application domain and send to platformService endpoint to processing import.
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using (var scope = ServiceProvider.CreateScope())
        {
            var platformServiceClient = scope.ServiceProvider.GetRequiredService<IPlatformClient>();
            await platformServiceClient.MetadataImportAsync("Resources.PlatformServiceMetadata.json", typeof(Startup).Assembly, stoppingToken);
        }
    }
}

Add register your metadata importer service into DI

public static class Startup
{
//...
    public void ConfigureServices(IServiceCollection services)
    {
//...
        services.AddHostedService<MetadataImporterService>();
//...
    }
//...
}