diff --git a/.gitignore b/.gitignore index f6904bc1da..bd14707eef 100644 --- a/.gitignore +++ b/.gitignore @@ -38,6 +38,7 @@ bld/ # Visual Studio 2015/2017 cache/options directory .vs/ +.cr/ # Uncomment if you have tasks that create the project's static files in wwwroot #wwwroot/ diff --git a/docs/media/foundationallm-highlevel-architecture.png b/docs/media/foundationallm-highlevel-architecture.png index a6e1f27167..856843b768 100644 Binary files a/docs/media/foundationallm-highlevel-architecture.png and b/docs/media/foundationallm-highlevel-architecture.png differ diff --git a/docs/release-notes/breaking-changes.md b/docs/release-notes/breaking-changes.md index 5c83ca97d5..73a1eb72dc 100644 --- a/docs/release-notes/breaking-changes.md +++ b/docs/release-notes/breaking-changes.md @@ -53,6 +53,13 @@ Authorization API | `FoundationaLLM-Authorization-API` | `api://FoundationaLLM-A User Portal | `FoundationaLLM-Core-Portal` | `api://FoundationaLLM-Core-Portal` | N/A Management Portal | `FoundationaLLM-Management-Portal` | `api://FoundationaLLM-Management-Portal` | N/A +#### Changes in app configuration settings + +The `FoundationaLLM:APIs` and `FoundationaLLM:ExternalAPIs` configuration namespaces have been replaced with the `FoundationaLLM:APIEndpoints` configuration namespace. + +> [!IMPORTANT] +> All existing API registrations need to be updated to reflect these changes. The only setting that will exist under `FoundationaLLM:APIEndpoints` is `APIKey` (for those API enpoints which use API key authentication), all the other settings are now part of the `APIEndpoint` artifact managed by the `FoundationaLLM.Configuration` resource provider. + ### Pre-0.8.0 1. Vectorization resource stores use a unique collection name, `Resources`. They also add a new top-level property named `DefaultResourceName`. diff --git a/src/FoundationaLLM.sln b/src/FoundationaLLM.sln index 768515e1ec..8faaad2998 100644 --- a/src/FoundationaLLM.sln +++ b/src/FoundationaLLM.sln @@ -113,6 +113,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "ManagementClient", "dotnet\ EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Management.Client.Tests", "..\tests\dotnet\Management.Client.Tests\Management.Client.Tests.csproj", "{2B369949-0297-485E-9455-E8F54D078DB3}" EndProject +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "AIModel", "dotnet\AIModel\AIModel.csproj", "{58D9C40B-3BE6-42CC-9FAB-ECF20FDA2E84}" +EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GatewayAdapterAPI", "dotnet\GatewayAdapterAPI\GatewayAdapterAPI.csproj", "{F826A354-9DF5-4DE5-97CB-F8F0D4566C0F}" EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "State", "dotnet\State\State.csproj", "{BF7614CE-4E64-4E69-A56D-A7E72EDA8ABD}" @@ -292,6 +294,10 @@ Global {2B369949-0297-485E-9455-E8F54D078DB3}.Debug|Any CPU.Build.0 = Debug|Any CPU {2B369949-0297-485E-9455-E8F54D078DB3}.Release|Any CPU.ActiveCfg = Release|Any CPU {2B369949-0297-485E-9455-E8F54D078DB3}.Release|Any CPU.Build.0 = Release|Any CPU + {58D9C40B-3BE6-42CC-9FAB-ECF20FDA2E84}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {58D9C40B-3BE6-42CC-9FAB-ECF20FDA2E84}.Debug|Any CPU.Build.0 = Debug|Any CPU + {58D9C40B-3BE6-42CC-9FAB-ECF20FDA2E84}.Release|Any CPU.ActiveCfg = Release|Any CPU + {58D9C40B-3BE6-42CC-9FAB-ECF20FDA2E84}.Release|Any CPU.Build.0 = Release|Any CPU {F826A354-9DF5-4DE5-97CB-F8F0D4566C0F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {F826A354-9DF5-4DE5-97CB-F8F0D4566C0F}.Debug|Any CPU.Build.0 = Debug|Any CPU {F826A354-9DF5-4DE5-97CB-F8F0D4566C0F}.Release|Any CPU.ActiveCfg = Release|Any CPU @@ -358,6 +364,7 @@ Global {95676184-9498-465C-B4FE-C67AA09A0397} = {C757E76C-AE42-4700-A92C-ED321D9CDB39} {69E631C8-3266-4D40-9A7A-D67C07972BE2} = {B6DC1190-2873-44A3-85B3-63D7BDE99231} {2B369949-0297-485E-9455-E8F54D078DB3} = {C757E76C-AE42-4700-A92C-ED321D9CDB39} + {58D9C40B-3BE6-42CC-9FAB-ECF20FDA2E84} = {B6DC1190-2873-44A3-85B3-63D7BDE99231} {F826A354-9DF5-4DE5-97CB-F8F0D4566C0F} = {B6DC1190-2873-44A3-85B3-63D7BDE99231} {BF7614CE-4E64-4E69-A56D-A7E72EDA8ABD} = {B6DC1190-2873-44A3-85B3-63D7BDE99231} {B0D4EBB4-2057-4081-8A80-65390BFBA654} = {B6DC1190-2873-44A3-85B3-63D7BDE99231} diff --git a/src/dotnet/AIModel/AIModel.csproj b/src/dotnet/AIModel/AIModel.csproj new file mode 100644 index 0000000000..cdd3b3b8c8 --- /dev/null +++ b/src/dotnet/AIModel/AIModel.csproj @@ -0,0 +1,16 @@ + + + + net8.0 + enable + enable + FoundationaLLM.AIModel + FoundationaLLM.AIModel + True + + + + + + + diff --git a/src/dotnet/AIModel/Models/AIModelReference.cs b/src/dotnet/AIModel/Models/AIModelReference.cs new file mode 100644 index 0000000000..6e5f70159a --- /dev/null +++ b/src/dotnet/AIModel/Models/AIModelReference.cs @@ -0,0 +1,27 @@ +using FoundationaLLM.Common.Constants.ResourceProviders; +using FoundationaLLM.Common.Exceptions; +using FoundationaLLM.Common.Models.ResourceProviders; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using System.Text.Json.Serialization; + +namespace FoundationaLLM.AIModel.Models +{ + /// + /// Contains a reference to an AIModel + /// + public class AIModelReference : ResourceReference + { + /// + /// The object type of the data source. + /// + [JsonIgnore] + public Type AIModelType => + Type switch + { + AIModelTypes.Basic => typeof(AIModelBase), + AIModelTypes.Completion => typeof(CompletionAIModel), + AIModelTypes.Embedding => typeof(EmbeddingAIModel), + _ => throw new ResourceProviderException($"The data source type {Type} is not supported.") + }; + } +} diff --git a/src/dotnet/AIModel/ResourceProviders/AIModelResourceProviderService.cs b/src/dotnet/AIModel/ResourceProviders/AIModelResourceProviderService.cs new file mode 100644 index 0000000000..25cfc93556 --- /dev/null +++ b/src/dotnet/AIModel/ResourceProviders/AIModelResourceProviderService.cs @@ -0,0 +1,403 @@ +using Azure.Messaging; +using FluentValidation; +using FoundationaLLM.AIModel.Models; +using FoundationaLLM.Common.Constants; +using FoundationaLLM.Common.Constants.Configuration; +using FoundationaLLM.Common.Constants.ResourceProviders; +using FoundationaLLM.Common.Exceptions; +using FoundationaLLM.Common.Interfaces; +using FoundationaLLM.Common.Models.Authentication; +using FoundationaLLM.Common.Models.Configuration.Instance; +using FoundationaLLM.Common.Models.Events; +using FoundationaLLM.Common.Models.ResourceProviders; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Common.Services.ResourceProviders; +using Microsoft.AspNetCore.Http; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using System.Collections.Concurrent; +using System.Text; +using System.Text.Json; + +namespace FoundationaLLM.AIModel.ResourceProviders +{ + /// + /// Implements the FoundationaLLM.AIModel resource provider. + /// + /// The options providing the with instance settings. + /// The providing authorization services. + /// The providing storage services. + /// The providing event services. + /// The providing the factory to create resource validators. + /// The of the main dependency injection container. + /// The used to provide loggers for logging. + public class AIModelResourceProviderService( + IOptions instanceOptions, + IAuthorizationService authorizationService, + [FromKeyedServices(DependencyInjectionKeys.FoundationaLLM_ResourceProvider_AIModel)] IStorageService storageService, + IEventService eventService, + IResourceValidatorFactory resourceValidatorFactory, + IServiceProvider serviceProvider, + ILoggerFactory loggerFactory) + : ResourceProviderServiceBase( + instanceOptions.Value, + authorizationService, + storageService, + eventService, + resourceValidatorFactory, + serviceProvider, + loggerFactory.CreateLogger(), + [ + EventSetEventNamespaces.FoundationaLLM_ResourceProvider_AIModel + ]) + { + /// + protected override Dictionary GetResourceTypes() => + AIModelResourceProviderMetadata.AllowedResourceTypes; + + private ConcurrentDictionary _aiModelReferences; + + /// + protected override string _name => ResourceProviderNames.FoundationaLLM_AIModel; + private const string AIMODEL_REFERENCES_FILE_NAME = "_aiModel-references.json"; + private const string AIMODEL_REFERENCES_FILE_PATH = + $"/{ResourceProviderNames.FoundationaLLM_AIModel}/{AIMODEL_REFERENCES_FILE_NAME}"; + + /// + protected override async Task InitializeInternal() + { + _logger.LogInformation("Starting to initialize the {ResourceProvider} resource provider...", _name); + + if (await _storageService.FileExistsAsync(_storageContainerName, AIMODEL_REFERENCES_FILE_PATH, default)) + { + var fileContent = await _storageService.ReadFileAsync( + _storageContainerName, + AIMODEL_REFERENCES_FILE_PATH, + default); + + var resourceReferenceStore = + JsonSerializer.Deserialize>( + Encoding.UTF8.GetString(fileContent.ToArray())); + + _aiModelReferences = new ConcurrentDictionary( + resourceReferenceStore!.ToDictionary()); + } + else + { + await _storageService.WriteFileAsync( + _storageContainerName, + AIMODEL_REFERENCES_FILE_PATH, + JsonSerializer.Serialize(new ResourceReferenceStore + { + ResourceReferences = [] + }), + default, + default); + } + + _logger.LogInformation("The {ResourceProvider} resource provider was successfully initialized.", _name); + } + + #region Support for Management API + + /// + protected override async Task GetResourcesAsync(ResourcePath resourcePath, UnifiedUserIdentity userIdentity) => + resourcePath.ResourceTypeInstances[0].ResourceType switch + { + AIModelResourceTypeNames.AIModels => await LoadAIModels(resourcePath.ResourceTypeInstances[0], userIdentity), + _ => throw new ResourceProviderException($"The resource type {resourcePath.ResourceTypeInstances[0].ResourceType} is not supported by the {_name} resource provider.", + StatusCodes.Status400BadRequest) + }; + + #region Helpers for GetResourcesAsyncInternal + + private async Task>> LoadAIModels(ResourceTypeInstance instance, UnifiedUserIdentity userIdentity) + { + var aiModels = new List(); + + if (instance.ResourceId == null) + { + aiModels = (await Task.WhenAll(_aiModelReferences.Values + .Where(ar => !ar.Deleted) + .Select(ar => LoadAIModel(ar)))) + .Where(a => a != null) + .Select(a => a!) + .ToList(); + + } + else + { + AIModelBase? aiModel; + if (!_aiModelReferences.TryGetValue(instance.ResourceId, out var aiModelReference)) + { + aiModel = await LoadAIModel(null, instance.ResourceId); + if (aiModel != null) + aiModels.Add(aiModel); + } + else + { + if (aiModelReference.Deleted) + throw new ResourceProviderException( + $"Could not locate the {instance.ResourceId} aiModel resource.", + StatusCodes.Status404NotFound); + + aiModel = await LoadAIModel(aiModelReference); + if (aiModel != null) + aiModels.Add(aiModel); + } + } + return aiModels.Select(aiModel => new ResourceProviderGetResult() { Resource = aiModel, Actions = [], Roles = [] }).ToList(); + } + + /// + private async Task LoadAIModel(AIModelReference? aiModelReference, string? resourceId = null) + { + if (aiModelReference != null || !string.IsNullOrWhiteSpace(resourceId)) + { + aiModelReference ??= new AIModelReference + { + Name = resourceId!, + Type = AIModelTypes.Basic, + Filename = $"/{_name}/{resourceId}.json", + Deleted = false + }; + + + if (await _storageService.FileExistsAsync(_storageContainerName, aiModelReference.Filename, default)) + { + var fileContent = await _storageService.ReadFileAsync(_storageContainerName, aiModelReference.Filename, default); + var aiModel = JsonSerializer.Deserialize( + Encoding.UTF8.GetString(fileContent.ToArray()), + aiModelReference.AIModelType, + base._serializerSettings) as AIModelBase + ?? throw new ResourceProviderException($"Failed to load the AI Model {aiModelReference.Name}.", + StatusCodes.Status400BadRequest); + + if (!string.IsNullOrWhiteSpace(resourceId)) + { + aiModelReference.Type = aiModel.Type!; + _aiModelReferences.AddOrUpdate(aiModelReference.Name, aiModelReference, (k, v) => aiModelReference); + } + + return aiModel; + } + + if (string.IsNullOrWhiteSpace(resourceId)) + { + // Remove the reference from the dictionary since the file does not exist. + _aiModelReferences.TryRemove(aiModelReference.Name, out _); + return null; + } + } + throw new ResourceProviderException($"Could not locate the {aiModelReference.Name} AI model resource.", + StatusCodes.Status404NotFound); + } + + #endregion + + /// + protected override async Task UpsertResourceAsync(ResourcePath resourcePath, string serializedResource, UnifiedUserIdentity userIdentity) => + resourcePath.ResourceTypeInstances[0].ResourceType switch + { + AIModelResourceTypeNames.AIModels => await UpdateAIModel(resourcePath, serializedResource, userIdentity), + _ => throw new ResourceProviderException($"The resource type {resourcePath.ResourceTypeInstances[0].ResourceType} is not supported by the {_name} resource provider.", + StatusCodes.Status400BadRequest) + }; + + #region Helpers for UpsertResourceAsync + + private async Task UpdateAIModel(ResourcePath resourcePath, string serializedAIModel, UnifiedUserIdentity userIdentity) + { + + var aiModel = JsonSerializer.Deserialize(serializedAIModel) + ?? throw new ResourceProviderException("The object definition is invalid.", + StatusCodes.Status400BadRequest); + + if (_aiModelReferences.TryGetValue(aiModel.Name!, out var existingAIModelReference) + && existingAIModelReference!.Deleted) + throw new ResourceProviderException($"The AI model resource {existingAIModelReference.Name} cannot be added or updated.", + StatusCodes.Status400BadRequest); + + if (resourcePath.ResourceTypeInstances[0].ResourceId != aiModel.Name) + throw new ResourceProviderException("The resource path does not match the object definition (name mismatch).", + StatusCodes.Status400BadRequest); + + var aiModelReference = new AIModelReference + { + Name = aiModel.Name!, + Type = aiModel.Type!, + Filename = $"/{_name}/{aiModel.Name}.json", + Deleted = false + }; + + aiModel.ObjectId = resourcePath.GetObjectId(_instanceSettings.Id, _name); + + var validator = _resourceValidatorFactory.GetValidator(aiModelReference.AIModelType); + if (validator is IValidator aiModelValidator) + { + var context = new ValidationContext(aiModel); + var validationResult = await aiModelValidator.ValidateAsync(context); + if (!validationResult.IsValid) + { + throw new ResourceProviderException($"Validation failed: {string.Join(", ", validationResult.Errors.Select(e => e.ErrorMessage))}", + StatusCodes.Status400BadRequest); + } + } + + if (existingAIModelReference == null) + aiModel.CreatedBy = userIdentity.UPN; + else + aiModel.UpdatedBy = userIdentity.UPN; + + await _storageService.WriteFileAsync( + _storageContainerName, + aiModelReference.Filename, + JsonSerializer.Serialize(aiModel, _serializerSettings), + default, + default); + + _aiModelReferences.AddOrUpdate(aiModelReference.Name, aiModelReference, (k, v) => aiModelReference); + + await _storageService.WriteFileAsync( + _storageContainerName, + AIMODEL_REFERENCES_FILE_PATH, + JsonSerializer.Serialize(ResourceReferenceStore.FromDictionary(_aiModelReferences.ToDictionary())), + default, + default); + + return new ResourceProviderUpsertResult + { + ObjectId = (aiModel as AIModelBase)!.ObjectId + }; + } + + private string GetFileExtension(string fileName) => + Path.GetExtension(fileName); + + #endregion + + /// +#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously + protected override async Task ExecuteActionAsync(ResourcePath resourcePath, string serializedAction, UnifiedUserIdentity userIdentity) => throw new NotImplementedException(); +#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously + + /// + protected override async Task DeleteResourceAsync(ResourcePath resourcePath, UnifiedUserIdentity userIdentity) + { + switch (resourcePath.ResourceTypeInstances.Last().ResourceType) + { + case AIModelResourceTypeNames.AIModels: + await DeleteAIModel(resourcePath.ResourceTypeInstances); + break; + default: + throw new ResourceProviderException($"The resource type {resourcePath.ResourceTypeInstances.Last().ResourceType} is not supported by the {_name} resource provider.", + StatusCodes.Status400BadRequest); + }; + } + + #region Helpers for DeleteResourceAsync + + private async Task DeleteAIModel(List instances) + { + if (_aiModelReferences.TryGetValue(instances.Last().ResourceId!, out var aiModelReference)) + { + if (!aiModelReference.Deleted) + { + aiModelReference.Deleted = true; + + await _storageService.WriteFileAsync( + _storageContainerName, + AIMODEL_REFERENCES_FILE_PATH, + JsonSerializer.Serialize(ResourceReferenceStore.FromDictionary(_aiModelReferences.ToDictionary())), + default, + default); + } + } + else + { + throw new ResourceProviderException($"Could not locate the {instances.Last().ResourceId} aiModel resource.", + StatusCodes.Status404NotFound); + } + } + + #endregion + + #endregion + + /// + protected override T GetResourceInternal(ResourcePath resourcePath) where T : class + { + if (resourcePath.ResourceTypeInstances.Count != 1) + throw new ResourceProviderException($"Invalid resource path"); + + if (typeof(T) != typeof(AIModelBase)) + throw new ResourceProviderException($"The type of requested resource ({typeof(T)}) does not match the resource type specified in the path ({resourcePath.ResourceTypeInstances[0].ResourceType})."); + + _aiModelReferences.TryGetValue(resourcePath.ResourceTypeInstances[0].ResourceId!, out var aiModelReference); + if (aiModelReference == null || aiModelReference.Deleted) + throw new ResourceProviderException($"The resource {resourcePath.ResourceTypeInstances[0].ResourceId!} of type {resourcePath.ResourceTypeInstances[0].ResourceType} was not found."); + + var aiModel = LoadAIModel(aiModelReference).Result; + return aiModel as T + ?? throw new ResourceProviderException($"The resource {resourcePath.ResourceTypeInstances[0].ResourceId!} of type {resourcePath.ResourceTypeInstances[0].ResourceType} was not found."); + } + + + #region Event handling + + /// + protected override async Task HandleEvents(EventSetEventArgs e) + { + _logger.LogInformation("{EventsCount} events received in the {EventsNamespace} events namespace.", + e.Events.Count, e.Namespace); + + switch (e.Namespace) + { + case EventSetEventNamespaces.FoundationaLLM_ResourceProvider_AIModel: + foreach (var @event in e.Events) + await HandleAIModelResourceProviderEvent(@event); + break; + default: + // Ignore sliently any event namespace that's of no interest. + break; + } + + await Task.CompletedTask; + } + + private async Task HandleAIModelResourceProviderEvent(CloudEvent e) + { + if (string.IsNullOrWhiteSpace(e.Subject)) + return; + + var fileName = e.Subject.Split("/").Last(); + + _logger.LogInformation("The file [{FileName}] managed by the [{ResourceProvider}] resource provider has changed and will be reloaded.", + fileName, _name); + + var aiModelReference = new AIModelReference + { + Name = Path.GetFileNameWithoutExtension(fileName), + Filename = $"/{_name}/{fileName}", + Type = nameof(AIModelBase), + Deleted = false + }; + + var aiModel = await LoadAIModel(aiModelReference); + aiModelReference.Name = aiModel.Name; + aiModelReference.Type = aiModel.Type!; + + _aiModelReferences.AddOrUpdate( + aiModelReference.Name, + aiModelReference, + (k, v) => v); + + _logger.LogInformation("The aiModel reference for the [{AIModelName}] agent or type [{AIModelType}] was loaded.", + aiModelReference.Name, aiModelReference.Type); + } + + #endregion + + } +} diff --git a/src/dotnet/AIModel/ResourceProviders/DependencyInjection.cs b/src/dotnet/AIModel/ResourceProviders/DependencyInjection.cs new file mode 100644 index 0000000000..213e0b09e0 --- /dev/null +++ b/src/dotnet/AIModel/ResourceProviders/DependencyInjection.cs @@ -0,0 +1,63 @@ +using FluentValidation; +using FoundationaLLM.AIModel.ResourceProviders; +using FoundationaLLM.AIModel.Validation; +using FoundationaLLM.Common.Constants.Configuration; +using FoundationaLLM.Common.Interfaces; +using FoundationaLLM.Common.Models.Configuration.Instance; +using FoundationaLLM.Common.Models.Configuration.Storage; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Common.Services.Storage; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Hosting; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; + +namespace FoundationaLLM +{ + /// + /// Data Source resource provider service implementation of resource provider dependency injection extensions. + /// + public static partial class DependencyInjection + { + /// + /// Add the AIModel resource provider and its related services the the dependency injection container. + /// + /// The application builder. + public static void AddAIModelResourceProvider(this IHostApplicationBuilder builder) + { + builder.Services.AddOptions( + DependencyInjectionKeys.FoundationaLLM_ResourceProvider_AIModel) + .Bind(builder.Configuration.GetSection(AppConfigurationKeySections.FoundationaLLM_AIModel_ResourceProviderService_Storage)); + + builder.Services.AddSingleton(sp => + { + var settings = sp.GetRequiredService>() + .Get(DependencyInjectionKeys.FoundationaLLM_ResourceProvider_AIModel); + var logger = sp.GetRequiredService>(); + + return new BlobStorageService( + Options.Create(settings), + logger) + { + InstanceName = DependencyInjectionKeys.FoundationaLLM_ResourceProvider_AIModel + }; + }); + + // Register validators. + builder.Services.AddSingleton, AIModelBaseValidator>(); + + builder.Services.AddSingleton(sp => + new AIModelResourceProviderService( + sp.GetRequiredService>(), + sp.GetRequiredService(), + sp.GetRequiredService>() + .Single(s => s.InstanceName == DependencyInjectionKeys.FoundationaLLM_ResourceProvider_AIModel), + sp.GetRequiredService(), + sp.GetRequiredService(), + sp, + sp.GetRequiredService())); + builder.Services.ActivateSingleton(); + } + } +} diff --git a/src/dotnet/AIModel/Validation/AIModelValidator.cs b/src/dotnet/AIModel/Validation/AIModelValidator.cs new file mode 100644 index 0000000000..f0d7f48171 --- /dev/null +++ b/src/dotnet/AIModel/Validation/AIModelValidator.cs @@ -0,0 +1,17 @@ +using FluentValidation; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Common.Validation.ResourceProvider; + +namespace FoundationaLLM.AIModel.Validation +{ + /// + /// Validator for the model. + /// + public class AIModelBaseValidator : AbstractValidator + { + /// + /// Configures the validation rules for the model. + /// + public AIModelBaseValidator() => Include(new ResourceBaseValidator()); + } +} diff --git a/src/dotnet/Authorization/Data/AuthorizableActions.json b/src/dotnet/Authorization/Data/AuthorizableActions.json index 9bbe57e995..e73b568bb4 100644 --- a/src/dotnet/Authorization/Data/AuthorizableActions.json +++ b/src/dotnet/Authorization/Data/AuthorizableActions.json @@ -283,6 +283,26 @@ "field_name": "FoundationaLLM_Attachment_Attachments_Delete" } ] + }, + { + "category": "AIModel", + "actions": [ + { + "name": "FoundationaLLM.AIModel/aiModels/read", + "description": "Read aiModels models", + "field_name": "FoundationaLLM_AIModel_Models_Read" + }, + { + "name": "FoundationaLLM.AIModel/aiModels/write", + "description": "Create or update models.", + "field_name": "FoundationaLLM_AIModel_Models_Write" + }, + { + "name": "FoundationaLLM.AIModel/aiModels/delete", + "description": "Delete models.", + "field_name": "FoundationaLLM_AIModel_Models_Delete" + } + ] } ] diff --git a/src/dotnet/Authorization/Models/AuthorizableActions.cs b/src/dotnet/Authorization/Models/AuthorizableActions.cs index 60e9f25fcd..5b1ef0ef43 100644 --- a/src/dotnet/Authorization/Models/AuthorizableActions.cs +++ b/src/dotnet/Authorization/Models/AuthorizableActions.cs @@ -355,6 +355,27 @@ public static class AuthorizableActions "Delete attachments.", "Attachment") }, + { + AuthorizableActionNames.FoundationaLLM_AIModel_AIModels_Read, + new AuthorizableAction( + AuthorizableActionNames.FoundationaLLM_AIModel_AIModels_Read, + "Read AIModels.", + "AIModel") + }, + { + AuthorizableActionNames.FoundationaLLM_AIModel_AIModels_Write, + new AuthorizableAction( + AuthorizableActionNames.FoundationaLLM_AIModel_AIModels_Write, + "Create or update AIModels.", + "AIModel") + }, + { + AuthorizableActionNames.FoundationaLLM_AIModel_AIModels_Delete, + new AuthorizableAction( + AuthorizableActionNames.FoundationaLLM_AIModel_AIModels_Delete, + "Delete AIModel.", + "AIModel") + }, }); /// diff --git a/src/dotnet/Authorization/Services/DependencyInjection.cs b/src/dotnet/Authorization/Services/DependencyInjection.cs index f2b0281169..d1dab4c312 100644 --- a/src/dotnet/Authorization/Services/DependencyInjection.cs +++ b/src/dotnet/Authorization/Services/DependencyInjection.cs @@ -3,6 +3,7 @@ using FoundationaLLM.Authorization.Models.Configuration; using FoundationaLLM.Authorization.Services; using FoundationaLLM.Authorization.Validation; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Constants.Configuration; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Authorization; @@ -31,7 +32,7 @@ public static void AddAuthorizationCore(this IHostApplicationBuilder builder) return new DataLakeStorageService( Options.Create(new BlobStorageServiceSettings { - AuthenticationType = BlobStorageAuthenticationTypes.AzureIdentity, + AuthenticationType = AuthenticationTypes.AzureIdentity, AccountName = builder.Configuration[KeyVaultSecretNames.FoundationaLLM_AuthorizationAPI_Storage_AccountName] }), sp.GetRequiredService>()) diff --git a/src/dotnet/Authorization/Utils/ResourcePathUtils.cs b/src/dotnet/Authorization/Utils/ResourcePathUtils.cs index 59f77480db..4b5632e582 100644 --- a/src/dotnet/Authorization/Utils/ResourcePathUtils.cs +++ b/src/dotnet/Authorization/Utils/ResourcePathUtils.cs @@ -76,6 +76,7 @@ private static Dictionary GetAllowedResourceType ResourceProviderNames.FoundationaLLM_Configuration => ConfigurationResourceProviderMetadata.AllowedResourceTypes, ResourceProviderNames.FoundationaLLM_Attachment => AttachmentResourceProviderMetadata.AllowedResourceTypes, ResourceProviderNames.FoundationaLLM_Authorization => AuthorizationResourceProviderMetadata.AllowedResourceTypes, + ResourceProviderNames.FoundationaLLM_AIModel => AIModelResourceProviderMetadata.AllowedResourceTypes, _ => [] }; } diff --git a/src/dotnet/Common/Constants/Agents/AgentParameterKeys.cs b/src/dotnet/Common/Constants/Agents/AgentParametersKeys.cs similarity index 74% rename from src/dotnet/Common/Constants/Agents/AgentParameterKeys.cs rename to src/dotnet/Common/Constants/Agents/AgentParametersKeys.cs index 16cb0f52c4..4d1378b582 100644 --- a/src/dotnet/Common/Constants/Agents/AgentParameterKeys.cs +++ b/src/dotnet/Common/Constants/Agents/AgentParametersKeys.cs @@ -3,7 +3,7 @@ /// /// Contains constants of the keys for all overridable Agent settings. /// - public static class AgentParameterKeys + public static class AgentParametersKeys { /// /// The key name for the index filter expression agent parameter. @@ -11,10 +11,18 @@ public static class AgentParameterKeys /// by the index. /// public const string IndexFilterExpression = "index_filter_expression"; + /// /// Controls the number of search results to return from an index for prompt augmentation. /// public const string IndexTopN = "index_top_n"; + /// + /// All agent parameter keys. + /// + public readonly static string[] All = [ + IndexFilterExpression, + IndexTopN + ]; } } diff --git a/src/dotnet/Common/Constants/Agents/CompletionRequestObjectsKeys.cs b/src/dotnet/Common/Constants/Agents/CompletionRequestObjectsKeys.cs new file mode 100644 index 0000000000..9051a9eeef --- /dev/null +++ b/src/dotnet/Common/Constants/Agents/CompletionRequestObjectsKeys.cs @@ -0,0 +1,23 @@ +using FoundationaLLM.Common.Models.Orchestration; + +namespace FoundationaLLM.Common.Constants.Agents +{ + /// + /// Contains constants for the keys that can be added to the dictionary. + /// + public static class CompletionRequestObjectsKeys + { + /// + /// The key name for the dictionary containing names and descriptions of agents other than the completion request's agent. + /// This value should be a dictionary where keys are agent names and values are agent descriptions. + /// + public const string AllAgents = "AllAgents"; + + /// + /// All completion request objects dictionary keys. + /// + public readonly static string[] All = [ + AllAgents + ]; + } +} diff --git a/src/dotnet/Common/Constants/Agents/EndpointConfigurationKeys.cs b/src/dotnet/Common/Constants/Agents/EndpointConfigurationKeys.cs deleted file mode 100644 index 98ba153c27..0000000000 --- a/src/dotnet/Common/Constants/Agents/EndpointConfigurationKeys.cs +++ /dev/null @@ -1,42 +0,0 @@ -namespace FoundationaLLM.Common.Constants.Agents -{ - /// - /// Contains constants of the keys for endpoint configuration settings. - /// - public static class EndpointConfigurationKeys - { - /// - /// The LLM provider. - /// Can be microsoft or openai. - /// - public const string Provider = "provider"; - - /// - /// The API Endpoint configuration setting. - /// This value should be a URI representing the endpoint of the API. - /// - public const string Endpoint = "endpoint"; - - /// - /// The API Key configuration setting. - /// - public const string APIKey = "api_key"; - - /// - /// The API Version configuration setting. - /// - public const string APIVersion = "api_version"; - - /// - /// The type of authentication to use for connecting to the endpoint. - /// This value with be either key or token. - /// - public const string AuthenticationType = "authentication_type"; - - /// - /// The type of operation the endpoint is performing. - /// This value should be completion or chat. - /// - public const string OperationType = "operation_type"; - } -} diff --git a/src/dotnet/Common/Constants/Agents/ModelParameterKeys.cs b/src/dotnet/Common/Constants/Agents/ModelParametersKeys.cs similarity index 83% rename from src/dotnet/Common/Constants/Agents/ModelParameterKeys.cs rename to src/dotnet/Common/Constants/Agents/ModelParametersKeys.cs index c8ec1ec350..f4afab8abb 100644 --- a/src/dotnet/Common/Constants/Agents/ModelParameterKeys.cs +++ b/src/dotnet/Common/Constants/Agents/ModelParametersKeys.cs @@ -3,29 +3,21 @@ /// /// Contains constants of the keys for all overridable model settings. /// - public static class ModelParameterKeys + public static class ModelParametersKeys { - /// - /// The key name for the deployment_name model parameter. - /// This value should be a string representing the name of the model deployment in Azure OpenAI. - /// - public const string DeploymentName = "deployment_name"; - /// - /// The key name for the model version parameter. - /// This value should be a string representing the version of the model to use. - /// - public const string Version = "version"; /// /// Controls randomness. Lowering the temperature means that the model will produce more repetitive and /// deterministic responses. Increasing the temperature will result in more unexpected or creative responses. /// Try adjusting temperature or Top P but not both. This value should be a float between 0.0 and 1.0. /// public const string Temperature = "temperature"; + /// /// The number of highest probability vocabulary tokens to keep for top-k-filtering. /// Default value is null, which disables top-k-filtering. /// public const string TopK = "top_k"; + /// /// The cumulative probability of parameter highest probability vocabulary tokens to keep for nucleus sampling. /// Top P (or Top Probabilities) is imilar to temperature, this controls randomness but uses a different method. @@ -33,25 +25,42 @@ public static class ModelParameterKeys /// choose from tokens with both high and low likelihood. Try adjusting temperature or Top P but not both. /// public const string TopP = "top_p"; + /// /// Whether or not to use sampling; use greedy decoding otherwise. /// public const string DoSample = "do_sample"; + /// /// Sets a limit on the number of tokens per model response. The API supports a maximum of 4000 tokens shared /// between the prompt (including system message, examples, message history, and user query) and the model's /// response. One token is roughly 4 characters for typical English text. /// public const string MaxNewTokens = "max_new_tokens"; + /// /// Whether or not to return the full text (prompt + response) or only the generated part (response). /// Default value is false. /// public const string ReturnFullText = "return_full_text"; + /// /// Whether to ignore the EOS token and continue generating tokens after the EOS token is generated. /// Defaults to False. /// public const string IgnoreEOS = "ignore_eos"; + + /// + /// All model parameter keys. + /// + public readonly static string[] All = [ + Temperature, + TopK, + TopP, + DoSample, + MaxNewTokens, + ReturnFullText, + IgnoreEOS + ]; } } diff --git a/src/dotnet/Common/Constants/Authentication/AuthenticationParametersKeys.cs b/src/dotnet/Common/Constants/Authentication/AuthenticationParametersKeys.cs new file mode 100644 index 0000000000..3f0b48d8a7 --- /dev/null +++ b/src/dotnet/Common/Constants/Authentication/AuthenticationParametersKeys.cs @@ -0,0 +1,37 @@ +namespace FoundationaLLM.Common.Constants.Authentication +{ + /// + /// Provides authetication key names for the AuthenticationParameters properties on ApiEndpointConfigurations + /// + public static class AuthenticationParametersKeys + { + /// + /// The name of the App Config entry that contains the API key. + /// + public const string APIKeyConfigurationName = "api_key_configuration_name"; + + /// + /// The scope required for authentication. + /// + public const string Scope = "scope"; + + /// + /// The name of the header where the API key should provided. + /// + public const string APIKeyHeaderName = "api_key_header_name"; + + /// + /// An optional prefix that must be added before the API key (e.g., Bearer). + /// + public const string APIKeyPrefix = "api_key_prefix"; + + /// + /// All authentication parameter keys. + /// + public readonly static string[] All = [ + APIKeyConfigurationName, + APIKeyHeaderName, + Scope + ]; + } +} diff --git a/src/dotnet/Common/Models/Configuration/Storage/BlobStorageAuthenticationTypes.cs b/src/dotnet/Common/Constants/Authentication/AuthenticationTypes.cs similarity index 64% rename from src/dotnet/Common/Models/Configuration/Storage/BlobStorageAuthenticationTypes.cs rename to src/dotnet/Common/Constants/Authentication/AuthenticationTypes.cs index 9befbb53aa..81d65510ad 100644 --- a/src/dotnet/Common/Models/Configuration/Storage/BlobStorageAuthenticationTypes.cs +++ b/src/dotnet/Common/Constants/Authentication/AuthenticationTypes.cs @@ -1,15 +1,9 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Text; -using System.Threading.Tasks; - -namespace FoundationaLLM.Common.Models.Configuration.Storage +namespace FoundationaLLM.Common.Constants.Authentication { /// - /// Types of authentication for blob storage accounts. + /// Authentication types for API Endpoints /// - public enum BlobStorageAuthenticationTypes + public enum AuthenticationTypes { /// /// Unknown authentication type. @@ -21,6 +15,11 @@ public enum BlobStorageAuthenticationTypes /// AzureIdentity, + /// + /// API key authentication type. + /// + APIKey, + /// /// Connection string authentication type. /// diff --git a/src/dotnet/Common/Constants/Authorization/AuthorizableActionNames.cs b/src/dotnet/Common/Constants/Authorization/AuthorizableActionNames.cs index bc3d29fcea..d1948f0e9b 100644 --- a/src/dotnet/Common/Constants/Authorization/AuthorizableActionNames.cs +++ b/src/dotnet/Common/Constants/Authorization/AuthorizableActionNames.cs @@ -281,5 +281,24 @@ public static class AuthorizableActionNames public const string FoundationaLLM_Attachment_Attachments_Delete = "FoundationaLLM.Attachment/attachments/delete"; #endregion + + #region AIModel + + /// + /// Read aiModels models + /// + public const string FoundationaLLM_AIModel_AIModels_Read = "FoundationaLLM.AIModel/aiModels/read"; + + /// + /// Create or update models. + /// + public const string FoundationaLLM_AIModel_AIModels_Write = "FoundationaLLM.AIModel/aiModels/write"; + + /// + /// Delete models. + /// + public const string FoundationaLLM_AIModel_AIModels_Delete = "FoundationaLLM.AIModel/aiModels/delete"; + + #endregion } } diff --git a/src/dotnet/Common/Constants/Agents/LanguageModelProviders.cs b/src/dotnet/Common/Constants/Configuration/APIEndpointProviders.cs similarity index 81% rename from src/dotnet/Common/Constants/Agents/LanguageModelProviders.cs rename to src/dotnet/Common/Constants/Configuration/APIEndpointProviders.cs index 3f536e618d..2ffd36caba 100644 --- a/src/dotnet/Common/Constants/Agents/LanguageModelProviders.cs +++ b/src/dotnet/Common/Constants/Configuration/APIEndpointProviders.cs @@ -1,9 +1,9 @@ -namespace FoundationaLLM.Common.Constants.Agents +namespace FoundationaLLM.Common.Constants.Configuration { /// /// Language Model provider constants. /// - public static class LanguageModelProviders + public static class APIEndpointProviders { /// /// Microsoft diff --git a/src/dotnet/Common/Constants/Configuration/AppConfigurationKeys.cs b/src/dotnet/Common/Constants/Configuration/AppConfigurationKeys.cs index 3d561874e9..4289ba6bd1 100644 --- a/src/dotnet/Common/Constants/Configuration/AppConfigurationKeys.cs +++ b/src/dotnet/Common/Constants/Configuration/AppConfigurationKeys.cs @@ -860,9 +860,9 @@ public static class AppConfigurationKeyFilters /// public const string FoundationaLLM_APIs = "FoundationaLLM:APIs:*"; /// - /// The key filter for the FoundationaLLM:ExternalAPIs:* app configuration settings. + /// The key filter for the FoundationaLLM:APIEndpoints:* app configuration settings. /// - public const string FoundationaLLM_ExternalAPIs = "FoundationaLLM:ExternalAPIs:*"; + public const string FoundationaLLM_APIEndpoints = "FoundationaLLM:APIEndpoints:*"; /// /// The key filter for the FoundationaLLM:APIs:VectorizationAPI:* app configuration settings. /// @@ -936,6 +936,10 @@ public static class AppConfigurationKeyFilters /// public const string FoundationaLLM_Prompt = "FoundationaLLM:Prompt:*"; /// + /// The key filter for the FoundationaLLM:Prompt:* app configuration settings. + /// + public const string FoundationaLLM_AIModel = "FoundationaLLM:AIModel:*"; + /// /// The key filter for the FoundationaLLM:Events:* app configuration settings. /// public const string FoundationaLLM_Events = "FoundationaLLM:Events:*"; @@ -1054,9 +1058,9 @@ public static class AppConfigurationKeySections /// public const string FoundationaLLM_APIs_StateAPI = "FoundationaLLM:APIs:StateAPI"; /// - /// The key section for the FoundationaLLM:ExternalAPIs app configuration settings. + /// The key section for the FoundationaLLM:APIEndpoints app configuration settings. /// - public const string FoundationaLLM_ExternalAPIs = "FoundationaLLM:ExternalAPIs"; + public const string FoundationaLLM_APIEndpoints = "FoundationaLLM:APIEndpoints"; /// /// The key section for the FoundationaLLM:Orchestration app configuration settings. /// @@ -1134,6 +1138,9 @@ public static class AppConfigurationKeySections /// public const string FoundationaLLM_AzureAIStudio_BlobStorageServiceSettings = "FoundationaLLM:AzureAIStudio:BlobStorageServiceSettings"; /// + /// The key section for the FoundationaLLM:AIModels app configuration settings. + /// + public const string FoundationaLLM_AIModels = "FoundationaLLM:AIModels"; /// The key section for the FoundationaLLM:State app configuration settings. /// public const string FoundationaLLM_State = "FoundationaLLM:State"; @@ -1169,6 +1176,11 @@ public static class AppConfigurationKeySections /// The key section for the FoundationaLLM:Attachment:ResourceProviderService:Storage app configuration settings. /// public const string FoundationaLLM_Attachment_ResourceProviderService_Storage = "FoundationaLLM:Attachment:ResourceProviderService:Storage"; + + /// + /// The key section for the FoundationaLLM:Attachment:ResourceProviderService:Storage app configuration settings. + /// + public const string FoundationaLLM_AIModel_ResourceProviderService_Storage = "FoundationaLLM:AIModel:ResourceProviderService:Storage"; #endregion #region Event Grid events diff --git a/src/dotnet/Common/Constants/Configuration/DependencyInjectionKeys.cs b/src/dotnet/Common/Constants/Configuration/DependencyInjectionKeys.cs index 789db5afa7..0fe52c821c 100644 --- a/src/dotnet/Common/Constants/Configuration/DependencyInjectionKeys.cs +++ b/src/dotnet/Common/Constants/Configuration/DependencyInjectionKeys.cs @@ -86,11 +86,17 @@ public static class DependencyInjectionKeys /// The dependency injection key for the FoundationaLLM.Attachment resource provider. /// public const string FoundationaLLM_ResourceProvider_Attachment = "FoundationaLLM:ResourceProvider:Attachment"; + /// /// The dependency injection key for the FoundationaLLM.Authorization resource provider. /// public const string FoundationaLLM_ResourceProvider_Authorization = "FoundationaLLM:ResourceProvider:Authorization"; + /// + /// The dependency injection key for the FoundationaLLM.AIModel resource provider. + /// + public const string FoundationaLLM_ResourceProvider_AIModel = "FoundationaLLM:ResourceProvider:AIModel"; + #endregion } } diff --git a/src/dotnet/Common/Constants/EventSetEventNamespaces.cs b/src/dotnet/Common/Constants/EventSetEventNamespaces.cs index 7825545fcb..559b46fc96 100644 --- a/src/dotnet/Common/Constants/EventSetEventNamespaces.cs +++ b/src/dotnet/Common/Constants/EventSetEventNamespaces.cs @@ -31,9 +31,14 @@ public static class EventSetEventNamespaces /// public const string FoundationaLLM_ResourceProvider_DataSource = "ResourceProvider.FoundationaLLM.DataSource"; - /// + /// /// The namespace name for events concerning the FoundationaLLM.Attachment resource provider. /// public const string FoundationaLLM_ResourceProvider_Attachment = "ResourceProvider.FoundationaLLM.Attachment"; + + /// + /// The namespace name for events concerning the FoundationaLLM.AIModel resource provider. + /// + public const string FoundationaLLM_ResourceProvider_AIModel = "ResourceProvider.FoundationaLLM.AIModel"; } } diff --git a/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceProviderActions.cs b/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceProviderActions.cs new file mode 100644 index 0000000000..387f4d72df --- /dev/null +++ b/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceProviderActions.cs @@ -0,0 +1,18 @@ +namespace FoundationaLLM.Common.Constants.ResourceProviders +{ + /// + /// The names of the actions implemented by the AIModel resource provider. + /// + public class AIModelResourceProviderActions + { + /// + /// Check the validity of a resource name. + /// + public const string CheckName = "checkname"; + /// + /// Purges a soft-deleted resource. + /// + public const string Purge = "purge"; + + } +} diff --git a/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceProviderMetadata.cs b/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceProviderMetadata.cs new file mode 100644 index 0000000000..00de79432a --- /dev/null +++ b/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceProviderMetadata.cs @@ -0,0 +1,38 @@ +using FoundationaLLM.Common.Models.ResourceProviders; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; + +namespace FoundationaLLM.Common.Constants.ResourceProviders +{ + /// + /// Provides metadata for the FoundationaLLM.AIModel resource provider. + /// + public static class AIModelResourceProviderMetadata + { + /// + /// The metadata describing the resource types allowed by the resource provider. + /// + public static Dictionary AllowedResourceTypes => new() + { + { + AIModelResourceTypeNames.AIModels, + new ResourceTypeDescriptor( + AIModelResourceTypeNames.AIModels) + { + AllowedTypes = [ + new ResourceTypeAllowedTypes(HttpMethod.Get.Method, [], [], [typeof(ResourceProviderGetResult)]), + new ResourceTypeAllowedTypes(HttpMethod.Post.Method, [], [typeof(AIModelBase)], [typeof(ResourceProviderUpsertResult)]), + new ResourceTypeAllowedTypes(HttpMethod.Delete.Method, [], [], []), + ], + Actions = [ + new ResourceTypeAction(DataSourceResourceProviderActions.CheckName, false, true, [ + new ResourceTypeAllowedTypes(HttpMethod.Post.Method, [], [typeof(ResourceName)], [typeof(ResourceNameCheckResult)]) + ]), + new ResourceTypeAction(DataSourceResourceProviderActions.Purge, true, false, [ + new ResourceTypeAllowedTypes(HttpMethod.Post.Method, [], [], [typeof(ResourceProviderActionResult)]) + ]) + ] + } + } + }; + } +} diff --git a/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceTypeNames.cs b/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceTypeNames.cs new file mode 100644 index 0000000000..052fd5c837 --- /dev/null +++ b/src/dotnet/Common/Constants/ResourceProviders/AIModelResourceTypeNames.cs @@ -0,0 +1,13 @@ +namespace FoundationaLLM.Common.Constants.ResourceProviders +{ + /// + /// Contains constants of the names of the resource types managed by the FoundationaLLM.AIModel resource provider. + /// + public static class AIModelResourceTypeNames + { + /// + /// AIModels. + /// + public const string AIModels = "aiModels"; + } +} diff --git a/src/dotnet/Common/Constants/ResourceProviders/AIModelTypes.cs b/src/dotnet/Common/Constants/ResourceProviders/AIModelTypes.cs new file mode 100644 index 0000000000..e485ed3b1c --- /dev/null +++ b/src/dotnet/Common/Constants/ResourceProviders/AIModelTypes.cs @@ -0,0 +1,24 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Threading.Tasks; + +namespace FoundationaLLM.Common.Constants.ResourceProviders +{ + public static class AIModelTypes + { + /// + /// Basic AIModel type without practical functionality. Used as base for all other model types. + /// + public const string Basic = "basic"; + /// + /// Embedding model type + /// + public const string Embedding = "embedding"; + /// + /// Completion model type + /// + public const string Completion = "completion"; + } +} diff --git a/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceProviderMetadata.cs b/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceProviderMetadata.cs index b26a7b9457..ab703f8156 100644 --- a/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceProviderMetadata.cs +++ b/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceProviderMetadata.cs @@ -33,13 +33,13 @@ public static class ConfigurationResourceProviderMetadata } , { - ConfigurationResourceTypeNames.APIEndpoints, + ConfigurationResourceTypeNames.APIEndpointConfigurations, new ResourceTypeDescriptor( - ConfigurationResourceTypeNames.APIEndpoints) + ConfigurationResourceTypeNames.APIEndpointConfigurations) { AllowedTypes = [ - new ResourceTypeAllowedTypes(HttpMethod.Get.Method, [], [], [typeof(ResourceProviderGetResult)]), - new ResourceTypeAllowedTypes(HttpMethod.Post.Method, [], [typeof(APIEndpoint)], [typeof(ResourceProviderUpsertResult)]), + new ResourceTypeAllowedTypes(HttpMethod.Get.Method, [], [], [typeof(ResourceProviderGetResult)]), + new ResourceTypeAllowedTypes(HttpMethod.Post.Method, [], [typeof(APIEndpointConfiguration)], [typeof(ResourceProviderUpsertResult)]), new ResourceTypeAllowedTypes(HttpMethod.Delete.Method, [], [], []), ] } diff --git a/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceTypeNames.cs b/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceTypeNames.cs index 4775e48a66..2c344c52d8 100644 --- a/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceTypeNames.cs +++ b/src/dotnet/Common/Constants/ResourceProviders/ConfigurationResourceTypeNames.cs @@ -13,6 +13,6 @@ public static class ConfigurationResourceTypeNames /// /// API Endpoint key values. /// - public const string APIEndpoints = "apiEndpoints"; + public const string APIEndpointConfigurations = "apiEndpointConfigurations"; } } diff --git a/src/dotnet/Common/Constants/ResourceProviders/ConfigurationTypes.cs b/src/dotnet/Common/Constants/ResourceProviders/ConfigurationTypes.cs index c666fb914a..52e02ec51e 100644 --- a/src/dotnet/Common/Constants/ResourceProviders/ConfigurationTypes.cs +++ b/src/dotnet/Common/Constants/ResourceProviders/ConfigurationTypes.cs @@ -23,6 +23,6 @@ public class ConfigurationTypes /// /// Api endpoint resource. /// - public const string APIEndpoint = "api-endpoint"; + public const string APIEndpointConfiguration = "api-endpoint"; } } diff --git a/src/dotnet/Common/Constants/ResourceProviders/ResourceProviderNames.cs b/src/dotnet/Common/Constants/ResourceProviders/ResourceProviderNames.cs index 7f8425902b..0a127d2328 100644 --- a/src/dotnet/Common/Constants/ResourceProviders/ResourceProviderNames.cs +++ b/src/dotnet/Common/Constants/ResourceProviders/ResourceProviderNames.cs @@ -42,6 +42,11 @@ public static class ResourceProviderNames /// public const string FoundationaLLM_Authorization = "FoundationaLLM.Authorization"; + /// + /// The name of the FoundationaLLM.AIModel resource provider. + /// + public const string FoundationaLLM_AIModel = "FoundationaLLM.AIModel"; + /// /// Contains all the resource provider names. /// @@ -52,6 +57,7 @@ public static class ResourceProviderNames FoundationaLLM_Prompt, FoundationaLLM_DataSource, FoundationaLLM_Attachment, - FoundationaLLM_Authorization]; + FoundationaLLM_Authorization, + FoundationaLLM_AIModel]; } } diff --git a/src/dotnet/Common/Models/Configuration/Events/AzureEventGridAuthenticationTypes.cs b/src/dotnet/Common/Models/Configuration/Events/AzureEventGridAuthenticationTypes.cs deleted file mode 100644 index 7fdd16c5e3..0000000000 --- a/src/dotnet/Common/Models/Configuration/Events/AzureEventGridAuthenticationTypes.cs +++ /dev/null @@ -1,29 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Text; -using System.Threading.Tasks; - -namespace FoundationaLLM.Common.Models.Configuration.Events -{ - /// - /// Types of authentication for Azure Event Grid namespaces. - /// - public enum AzureEventGridAuthenticationTypes - { - /// - /// Unknown authentication type. - /// - Unknown = 0, - - /// - /// Azure managed identity authentication type. - /// - AzureIdentity, - - /// - /// API key authentication type. - /// - APIKey - } -} diff --git a/src/dotnet/Common/Models/Configuration/Events/AzureEventGridEventServiceSettings.cs b/src/dotnet/Common/Models/Configuration/Events/AzureEventGridEventServiceSettings.cs index 3fd967fa36..81771713b5 100644 --- a/src/dotnet/Common/Models/Configuration/Events/AzureEventGridEventServiceSettings.cs +++ b/src/dotnet/Common/Models/Configuration/Events/AzureEventGridEventServiceSettings.cs @@ -1,4 +1,5 @@ -using FoundationaLLM.Common.Services.Events; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Services.Events; using System.Text.Json.Serialization; namespace FoundationaLLM.Common.Models.Configuration.Events @@ -9,10 +10,10 @@ namespace FoundationaLLM.Common.Models.Configuration.Events public class AzureEventGridEventServiceSettings { /// - /// A value indicating the type of authentication used. + /// A value indicating the type of authentication used. /// [JsonConverter(typeof(JsonStringEnumConverter))] - public required AzureEventGridAuthenticationTypes AuthenticationType { get; set; } + public required AuthenticationTypes AuthenticationType { get; set; } /// /// The Azure Event Grid namespace endpoint. diff --git a/src/dotnet/Common/Models/Configuration/Storage/BlobStorageServiceSettings.cs b/src/dotnet/Common/Models/Configuration/Storage/BlobStorageServiceSettings.cs index 37d014f781..4b66c2035f 100644 --- a/src/dotnet/Common/Models/Configuration/Storage/BlobStorageServiceSettings.cs +++ b/src/dotnet/Common/Models/Configuration/Storage/BlobStorageServiceSettings.cs @@ -1,4 +1,5 @@ -using System.Text.Json.Serialization; +using FoundationaLLM.Common.Constants.Authentication; +using System.Text.Json.Serialization; namespace FoundationaLLM.Common.Models.Configuration.Storage { @@ -8,10 +9,10 @@ namespace FoundationaLLM.Common.Models.Configuration.Storage public record BlobStorageServiceSettings { /// - /// A value indicating the type of authentication used. + /// A value indicating the type of authentication used. /// [JsonConverter(typeof(JsonStringEnumConverter))] - public required BlobStorageAuthenticationTypes AuthenticationType { get; set; } + public required AuthenticationTypes AuthenticationType { get; set; } /// /// The name of the blob storage account. diff --git a/src/dotnet/Common/Models/Orchestration/CompletionRequest.cs b/src/dotnet/Common/Models/Orchestration/CompletionRequest.cs index bebe95e23a..4917221402 100644 --- a/src/dotnet/Common/Models/Orchestration/CompletionRequest.cs +++ b/src/dotnet/Common/Models/Orchestration/CompletionRequest.cs @@ -1,43 +1,18 @@ -using FoundationaLLM.Common.Models.Chat; -using System.Text.Json.Serialization; +using System.Text.Json.Serialization; namespace FoundationaLLM.Common.Models.Orchestration; /// /// The completion request object. /// -public class CompletionRequest +public class CompletionRequest : CompletionRequestBase { - /// - /// The message history associated with the completion request. - /// - [JsonPropertyName("message_history")] - public List? MessageHistory { get; set; } = []; - - /// - /// The session ID. - /// - [JsonPropertyName("session_id")] - public string? SessionId { get; set; } - - /// - /// Represent the input or user prompt. - /// - [JsonPropertyName("user_prompt")] - public required string UserPrompt { get; set; } - /// /// The name of the selected agent. /// [JsonPropertyName("agent_name")] public string? AgentName { get; set; } - /// - /// One or more attachments to include with the orchestration request. - /// The values should be the ObjectID of the attachment(s). - /// - [JsonPropertyName("attachments")] - public List? Attachments { get; set; } /// /// A list of Gatekeeper feature names used by the orchestration request. @@ -46,7 +21,7 @@ public class CompletionRequest public string[]? GatekeeperOptions { get; set; } /// - /// Collection of model settings to override with the orchestration request. + /// Settings that override some aspects of behaviour of the orchestration. /// [JsonPropertyName("settings")] public OrchestrationSettings? Settings { get; set; } diff --git a/src/dotnet/Common/Models/Orchestration/CompletionRequestBase.cs b/src/dotnet/Common/Models/Orchestration/CompletionRequestBase.cs new file mode 100644 index 0000000000..1734a885e4 --- /dev/null +++ b/src/dotnet/Common/Models/Orchestration/CompletionRequestBase.cs @@ -0,0 +1,37 @@ +using FoundationaLLM.Common.Models.Chat; +using System.Text.Json.Serialization; + +namespace FoundationaLLM.Common.Models.Orchestration +{ + /// + /// Base for completion requests + /// + public class CompletionRequestBase + { + /// + /// The session ID. + /// + [JsonPropertyName("session_id")] + public string? SessionId { get; set; } + + /// + /// Represent the input or user prompt. + /// + [JsonPropertyName("user_prompt")] + public required string UserPrompt { get; init; } + + /// + /// The message history associated with the completion request. + /// + [JsonPropertyName("message_history")] + public List? MessageHistory { get; set; } = []; + + /// + /// One or more attachments to include with the orchestration request. + /// The values should be the ObjectID of the attachment(s). + /// + [JsonPropertyName("attachments")] + public List Attachments { get; init; } = []; + + } +} diff --git a/src/dotnet/Common/Models/Orchestration/CompletionResponse.cs b/src/dotnet/Common/Models/Orchestration/CompletionResponse.cs index bc481832e0..0d2638ac63 100644 --- a/src/dotnet/Common/Models/Orchestration/CompletionResponse.cs +++ b/src/dotnet/Common/Models/Orchestration/CompletionResponse.cs @@ -5,67 +5,8 @@ namespace FoundationaLLM.Common.Models.Orchestration; /// /// Response from a language model. /// -public class CompletionResponse +public class CompletionResponse : CompletionResponseBase { - /// - /// The completion response from the language model. - /// - [JsonPropertyName("completion")] - public string Completion { get; set; } - - /// - /// The citations used in building the completion response. - /// - [JsonPropertyName("citations")] - public Citation[]? Citations { get; set; } - - /// - /// The user prompt the language model responded to. - /// - [JsonPropertyName("user_prompt")] - public string UserPrompt { get; set; } - - /// - /// The full prompt composed by the LLM. - /// - [JsonPropertyName("full_prompt")] - public string? FullPrompt { get; set; } - - /// - /// The prompt template used by the LLM. - /// - [JsonPropertyName("prompt_template")] - public string? PromptTemplate { get; set; } - - /// - /// The name of the FoundationaLLM agent. - /// - [JsonPropertyName("agent_name")] - public string? AgentName { get; set; } - - /// - /// The number of tokens in the prompt. - /// - [JsonPropertyName("prompt_tokens")] - public int PromptTokens { get; set; } = 0; - - /// - /// The number of tokens in the completion. - /// - [JsonPropertyName("completion_tokens")] - public int CompletionTokens { get; set; } = 0; - - /// - /// The total number of tokens. - /// - [JsonPropertyName("total_tokens")] - public int TotalTokens => PromptTokens + CompletionTokens; - - /// - /// The total cost of executing the completion operation. - /// - [JsonPropertyName("total_cost")] - public float TotalCost { get; set; } = 0.0f; /// /// User prompt embedding. diff --git a/src/dotnet/Common/Models/Orchestration/CompletionResponseBase.cs b/src/dotnet/Common/Models/Orchestration/CompletionResponseBase.cs new file mode 100644 index 0000000000..af418c1bfe --- /dev/null +++ b/src/dotnet/Common/Models/Orchestration/CompletionResponseBase.cs @@ -0,0 +1,67 @@ +using System.Text.Json.Serialization; + +namespace FoundationaLLM.Common.Models.Orchestration +{ + public class CompletionResponseBase + { + /// + /// The completion response from the language model. + /// + [JsonPropertyName("completion")] + public string Completion { get; set; } + + /// + /// The citations used in building the completion response. + /// + [JsonPropertyName("citations")] + public Citation[]? Citations { get; set; } + + /// + /// The user prompt the language model responded to. + /// + [JsonPropertyName("user_prompt")] + public string UserPrompt { get; set; } + + /// + /// The full prompt composed by the LLM. + /// + [JsonPropertyName("full_prompt")] + public string? FullPrompt { get; set; } + + /// + /// The prompt template used by the LLM. + /// + [JsonPropertyName("prompt_template")] + public string? PromptTemplate { get; set; } + + /// + /// The name of the FoundationaLLM agent. + /// + [JsonPropertyName("agent_name")] + public string? AgentName { get; set; } + + /// + /// The number of tokens in the prompt. + /// + [JsonPropertyName("prompt_tokens")] + public int PromptTokens { get; set; } = 0; + + /// + /// The number of tokens in the completion. + /// + [JsonPropertyName("completion_tokens")] + public int CompletionTokens { get; set; } = 0; + + /// + /// The total number of tokens. + /// + [JsonPropertyName("total_tokens")] + public int TotalTokens => PromptTokens + CompletionTokens; + + /// + /// The total cost of executing the completion operation. + /// + [JsonPropertyName("total_cost")] + public float TotalCost { get; set; } = 0.0f; + } +} diff --git a/src/dotnet/Common/Models/Orchestration/Direct/AzureAICompletionParameters.cs b/src/dotnet/Common/Models/Orchestration/Direct/AzureAICompletionParameters.cs index a5d8918942..7cd817a949 100644 --- a/src/dotnet/Common/Models/Orchestration/Direct/AzureAICompletionParameters.cs +++ b/src/dotnet/Common/Models/Orchestration/Direct/AzureAICompletionParameters.cs @@ -1,5 +1,4 @@ using System.Text.Json.Serialization; -using FoundationaLLM.Common.Interfaces; namespace FoundationaLLM.Common.Models.Orchestration.Direct { diff --git a/src/dotnet/Common/Models/Orchestration/EndpointSettings.cs b/src/dotnet/Common/Models/Orchestration/EndpointSettings.cs deleted file mode 100644 index 9c38acca61..0000000000 --- a/src/dotnet/Common/Models/Orchestration/EndpointSettings.cs +++ /dev/null @@ -1,38 +0,0 @@ -using FoundationaLLM.Common.Constants.Agents; - -namespace FoundationaLLM.Common.Models.Orchestration -{ - /// - /// Settings for an orchestration endpoint. - /// - public class EndpointSettings - { - /// - /// Uri of the orchestration endpoint. - /// - public string? Endpoint { get; set; } - - /// - /// API key for authorizing against an endpoint. - /// - public string? APIKey { get; set; } - - /// - /// API version to use when accessing the endpoint. - /// - public string? APIVersion { get; set; } - - /// - /// The type of authentication to use against the endpoint. - /// This value should be either key or token. - /// - public string? AuthenticationType { get; set; } - - /// - /// Type of operation the endpoint is performing. - /// This value should be completions or chat. - /// Default value is chat. - /// - public string OperationType { get; set; } = OperationTypes.Chat; - } -} diff --git a/src/dotnet/Common/Models/Orchestration/LLMCompletionRequest.cs b/src/dotnet/Common/Models/Orchestration/LLMCompletionRequest.cs index e11d108af1..d132607f7f 100644 --- a/src/dotnet/Common/Models/Orchestration/LLMCompletionRequest.cs +++ b/src/dotnet/Common/Models/Orchestration/LLMCompletionRequest.cs @@ -1,47 +1,361 @@ -using FoundationaLLM.Common.Models.Chat; +using FoundationaLLM.Common.Constants.Agents; +using FoundationaLLM.Common.Constants.Configuration; +using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Models.ResourceProviders.Agent; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Common.Models.ResourceProviders.Configuration; +using FoundationaLLM.Common.Models.ResourceProviders.Prompt; +using FoundationaLLM.Common.Models.ResourceProviders.Vectorization; +using Microsoft.AspNetCore.Http; +using System.Text.Json; using System.Text.Json.Serialization; namespace FoundationaLLM.Common.Models.Orchestration { /// - /// LLM orchestration request + /// The completion request sent by the Orchestration API to any of the downstream orchestrator APIs. /// - public class LLMCompletionRequest + public class LLMCompletionRequest : CompletionRequestBase { + private bool _valid = false; + private bool _hasTextEmbeddingProfile = false; + private bool _hasIndexingProfiles = false; + + private AIModelBase? _aiModel; + private APIEndpointConfiguration? _aiModelEndpointConfiguration; + private Dictionary? _otherAgentsDescriptions; + private MultipartPrompt? _prompt; + private TextEmbeddingProfile? _textEmbeddingProfile; + private List? _indexingProfiles; + /// /// The agent that will process the completion request. /// public required AgentBase Agent { get; set; } /// - /// The session ID. + /// Dictionary of objects (indexed by names) resulting from exploding object identifiers in the Orchestration API. + /// + /// Can also contain objects that are not the direct result of exploding an object identifier. + /// + /// + /// The dictionary supports the following keys: + /// + /// + /// /instances/{instanceId}/providers/FoundationaLLM.Prompt/prompts/{name} + /// + /// + /// /instances/{instanceId}/providers/FoundationaLLM.AIModel/aiModels/{name} + /// + /// + /// /instances/{instanceId}/providers/FoundationaLLM.Configuration/apiEndpointConfigurations/{name} + /// + /// + /// /instances/{instanceId}/providers/FoundationaLLM.Vectorization/indexingProfiles/{name} + /// + /// + /// /instances/{instanceId}/providers/FoundationaLLM.Vectorization/textEmbeddingProfiles/{name} + /// + /// + /// AllAgents + /// + /// + /// + /// + [JsonPropertyName("objects")] + public Dictionary Objects { get; set; } = []; + + /// + /// Validates the content of this LLMCompletionRequest. + /// + /// + public void Validate() + { + // Avoid multiple validations. + if (_valid) return; + + if (Agent == null) + throw new OrchestrationException("The Agent property of the completion request cannot be null."); + + if (Agent.OrchestrationSettings == null) + throw new OrchestrationException("The OrchestrationSettings property of the agent cannot be null."); + + if (Objects == null) + throw new OrchestrationException("The Objects property of the completion request cannot be null."); + + if (string.IsNullOrWhiteSpace(Agent.AIModelObjectId)) + throw new OrchestrationException("Invalid AI model object id."); + + if (!Objects.TryGetValue( + Agent.AIModelObjectId, out var aiModelObject)) + throw new OrchestrationException("The AI model object is missing from the request's objects."); + + var aiModel = aiModelObject is JsonElement aiModelJsonElement + ? aiModelJsonElement.Deserialize() + : aiModelObject as AIModelBase; + + if (aiModel == null + || string.IsNullOrWhiteSpace(aiModel.EndpointObjectId) + || string.IsNullOrWhiteSpace(aiModel.DeploymentName) + || aiModel.ModelParameters == null) + throw new OrchestrationException("The AI model object provided in the request's objects is invalid."); + + if (!Objects.TryGetValue( + aiModel.EndpointObjectId, out var endpointObject)) + throw new OrchestrationException("The API endpoint configuration object is missing from the request's objects."); + + var endpoint = endpointObject is JsonElement endpointJsonElement + ? endpointJsonElement.Deserialize() + : endpointObject as APIEndpointConfiguration; + + if (endpoint == null + || string.IsNullOrWhiteSpace(endpoint.Provider) + || !APIEndpointProviders.All.Contains(endpoint.Provider) + || string.IsNullOrWhiteSpace(endpoint.Url)) + throw new OrchestrationException("The API endpoint configuration object provided in the request's objects is invalid."); + + if (string.IsNullOrWhiteSpace(Agent.PromptObjectId)) + throw new OrchestrationException("Invalid prompt object id."); + + if (!Objects.TryGetValue( + Agent.PromptObjectId, out var promptObject)) + throw new OrchestrationException("The prompt object is missing from the request's objects."); + + var prompt = promptObject is JsonElement promptJsonElement + ? promptJsonElement.Deserialize() + : promptObject as MultipartPrompt; + + if (prompt == null + || string.IsNullOrWhiteSpace(prompt.Prefix)) + throw new OrchestrationException("The prompt object provided in the request's objects is invalid."); + + if (Agent is KnowledgeManagementAgent kmAgent + && kmAgent.Vectorization != null) + { + if (!string.IsNullOrWhiteSpace(kmAgent.Vectorization.TextEmbeddingProfileObjectId)) + { + if (!Objects.TryGetValue( + kmAgent.Vectorization.TextEmbeddingProfileObjectId, out var textEmbeddingProfileObject)) + throw new OrchestrationException("The text embedding profile object is missing from the request's objects."); + + var textEmbeddingProfile = textEmbeddingProfileObject is JsonElement textEmbeddingProfileJsonElement + ? textEmbeddingProfileJsonElement.Deserialize() + : textEmbeddingProfileObject as TextEmbeddingProfile; + + if (textEmbeddingProfile == null + || textEmbeddingProfile.ConfigurationReferences == null + || !textEmbeddingProfile.ConfigurationReferences.TryGetValue("DeploymentName", out var deploymentNameConfigurationItem) + || string.IsNullOrWhiteSpace(deploymentNameConfigurationItem) + || !textEmbeddingProfile.ConfigurationReferences.TryGetValue("EndpointUrl", out var textEmbeddingEndpointConfigurationItem) + || string.IsNullOrWhiteSpace(textEmbeddingEndpointConfigurationItem)) + throw new OrchestrationException("The text embedding profile object provided in the request's objects is invalid."); + + _hasTextEmbeddingProfile = true; + } + + if ((kmAgent.Vectorization.IndexingProfileObjectIds ?? []).Count > 0) + { + for (int i = 0; i < kmAgent.Vectorization.IndexingProfileObjectIds!.Count; i++) + { + var indexingProfileObjectId = kmAgent.Vectorization.IndexingProfileObjectIds[i]; + + if (string.IsNullOrEmpty(indexingProfileObjectId)) + throw new OrchestrationException($"The indexing profile object id at index {i} is invalid."); + + if (!Objects.TryGetValue( + indexingProfileObjectId, out var indexingProfileObject)) + throw new OrchestrationException($"The indexing profile object with id {indexingProfileObjectId} is missing from the request's objects."); + + var indexingProfile = indexingProfileObject is JsonElement indexingProfileJsonElement + ? indexingProfileJsonElement.Deserialize() + : indexingProfileObject as IndexingProfile; + + if (indexingProfile == null + || indexingProfile.Settings == null + || !indexingProfile.Settings.TryGetValue("IndexName", out var indexName) + || string.IsNullOrWhiteSpace(indexName)) + throw new OrchestrationException($"The indexing profile object with id {indexingProfileObjectId} provided in the request's objects is invalid."); + } + + _hasIndexingProfiles = true; + } + } + + _valid = true; + } + + /// + /// The object from the Objects dictionary. Ensure the Validate method is called before accessing this property. + /// + /// This object is supposed to be added to the Objects dictionary by the instantiator of this request based on the object identifier set on the agent. + /// /// - [JsonPropertyName("session_id")] - public string? SessionId { get; set; } + [JsonIgnore] + public AIModelBase AIModel + { + get + { + if (_aiModel != null) + return _aiModel; + + EnsureIsValid(); + + var aiModelObject = Objects[Agent.AIModelObjectId!]; + _aiModel = aiModelObject is JsonElement aiModelJsonElement + ? aiModelJsonElement.Deserialize()! + : (aiModelObject as AIModelBase)!; + + return _aiModel; + } + } /// - /// Prompt entered by the user. + /// The object from the Objects dictionary corresponding to the object from the same dictionary. Ensure the Validate method is called before accessing this property. + /// + /// This object is supposed to be added to the Objects dictionary by the instantiator of this request based on the object identifier set on the AIModel object associated with the agent. + /// /// - [JsonPropertyName("user_prompt")] - public string? UserPrompt { get; set; } + [JsonIgnore] + public APIEndpointConfiguration AIModelEndpointConfiguration + { + get + { + if (_aiModelEndpointConfiguration != null) + return _aiModelEndpointConfiguration; + + EnsureIsValid(); + + var endpointObject = Objects[AIModel.EndpointObjectId!]; + + _aiModelEndpointConfiguration = endpointObject is JsonElement endpointJsonElement + ? endpointJsonElement.Deserialize()! + : (endpointObject as APIEndpointConfiguration)!; + + return _aiModelEndpointConfiguration; + } + } /// - /// The message history associated with the completion request. + /// The dictionary of other agent descriptions from the Objects dictionary. Ensure the Validate method is called before accessing this property. + /// + /// This object is supposed to be added to the Objects dictionary by the instantiator of this request based on the descriptions of all agents the calling identity has access to, except for the agent associated with the request. + /// /// - [JsonPropertyName("message_history")] - public List? MessageHistory { get; init; } = []; + [JsonIgnore] + public Dictionary OtherAgentsDescriptions + { + get + { + if (_otherAgentsDescriptions != null) + return _otherAgentsDescriptions; + + EnsureIsValid(); + + _otherAgentsDescriptions = + Objects.TryGetValue(CompletionRequestObjectsKeys.AllAgents, out var allAgentDescriptions) + ? allAgentDescriptions is JsonElement allAgentDescriptionsJsonElement + ? allAgentDescriptionsJsonElement.Deserialize>()! + : (allAgentDescriptions as Dictionary)! + : []; + + return _otherAgentsDescriptions; + } + } /// - /// Collection of model settings to override with the orchestration request. + /// The object from the Objects dictionary. Ensure the Validate method is called before accessing this property. + /// + /// This object is supposed to be added to the Objects dictionary by the instantiator of this request based on the object identifier set on the agent. + /// /// - [JsonPropertyName("settings")] - public OrchestrationSettings? Settings { get; set; } + [JsonIgnore] + public MultipartPrompt Prompt + { + get + { + if (_prompt != null) + return _prompt; + + EnsureIsValid(); + + var promptObject = Objects[Agent.PromptObjectId!]; + + _prompt = promptObject is JsonElement promptJsonElement + ? promptJsonElement.Deserialize()! + : (promptObject as MultipartPrompt)!; + + return _prompt; + } + } /// - /// The list of attachments associated with the request. + /// The object from the Objects dictionary. Ensure the Validate method is called before accessing this property. + /// + /// This object is supposed to be added to the Objects dictionary by the instantiator of this request based on the object identifier set on the agent's vectorization settings. + /// /// - [JsonPropertyName("attachments")] - public List Attachments { get; init; } = []; + [JsonIgnore] + public TextEmbeddingProfile? TextEmbeddingProfile + { + get + { + if (_textEmbeddingProfile != null) + return _textEmbeddingProfile; + + EnsureIsValid(); + + if (!_hasTextEmbeddingProfile) + return null; + + var textEmbeddingProfileObject = Objects[(Agent as KnowledgeManagementAgent)!.Vectorization.TextEmbeddingProfileObjectId!]; + + _textEmbeddingProfile = textEmbeddingProfileObject is JsonElement textEmbeddingProfileJsonElement + ? textEmbeddingProfileJsonElement.Deserialize()! + : (textEmbeddingProfileObject as TextEmbeddingProfile)!; + + return _textEmbeddingProfile; + } + } + + /// + /// The list of objects from the Objects dictionary. Ensure the Validate method is called before accessing this property. + /// + /// These objects are supposed to be added to the Objects dictionary by the instantiator of this request based on the object identifiers set on the agent's vectorization settings. + /// + /// + [JsonIgnore] + public List? IndexingProfiles + { + get + { + if (_indexingProfiles != null) + return _indexingProfiles; + + EnsureIsValid(); + + if (!_hasIndexingProfiles) + return null; + + _indexingProfiles = (Agent as KnowledgeManagementAgent)!.Vectorization.IndexingProfileObjectIds! + .Select(indexingProfileObjectId => + { + var indexingProfileObject = Objects[indexingProfileObjectId]; + + return indexingProfileObject is JsonElement indexingProfileJsonElement + ? indexingProfileJsonElement.Deserialize()! + : (indexingProfileObject as IndexingProfile)!; + }) + .ToList(); + + + return _indexingProfiles; + } + } + + private void EnsureIsValid() + { + if (_valid) + throw new OrchestrationException("The request is either invalid or has not been validated yet."); + } } } diff --git a/src/dotnet/Common/Models/Orchestration/LLMCompletionResponse.cs b/src/dotnet/Common/Models/Orchestration/LLMCompletionResponse.cs index ad32dbdc11..5f9c7f4070 100644 --- a/src/dotnet/Common/Models/Orchestration/LLMCompletionResponse.cs +++ b/src/dotnet/Common/Models/Orchestration/LLMCompletionResponse.cs @@ -5,66 +5,7 @@ namespace FoundationaLLM.Common.Models.Orchestration /// /// LLMOrchestrationCompletionResponse class /// - public class LLMCompletionResponse + public class LLMCompletionResponse : CompletionResponseBase { - /// - /// The completion response from the orchestration engine. - /// - [JsonPropertyName("completion")] - public string? Completion { get; set; } - - /// - /// The citations used in building the completion response. - /// - [JsonPropertyName("citations")] - public Citation[]? Citations { get; set; } - - /// - /// The prompt received from the user. - /// - [JsonPropertyName("user_prompt")] - public string? UserPrompt { get; set; } - - /// - /// The full prompt composed by the LLM. - /// - [JsonPropertyName("full_prompt")] - public string? FullPrompt { get; set; } - - /// - /// The prompt template used by the LLM. - /// - [JsonPropertyName("prompt_template")] - public string? PromptTemplate { get; set; } - - /// - /// The name of the FoundationaLLM agent. - /// - [JsonPropertyName("agent_name")] - public string? AgentName { get; set; } - - /// - /// The number of tokens in the prompt. - /// - [JsonPropertyName("prompt_tokens")] - public int PromptTokens { get; set; } = 0; - - /// - /// The number of tokens in the completion. - /// - [JsonPropertyName("completion_tokens")] - public int CompletionTokens { get; set; } = 0; - - /// - /// Total tokens used. - /// - [JsonPropertyName("total_tokens")] - public int TotalTokens { get; set; } = 0; - - /// - /// Total cost of the completion execution. - /// - [JsonPropertyName("total_cost")] - public float TotalCost { get; set; } = 0f; } } diff --git a/src/dotnet/Common/Models/Orchestration/OrchestrationSettings.cs b/src/dotnet/Common/Models/Orchestration/OrchestrationSettings.cs index 19e4f1422d..befe3afdff 100644 --- a/src/dotnet/Common/Models/Orchestration/OrchestrationSettings.cs +++ b/src/dotnet/Common/Models/Orchestration/OrchestrationSettings.cs @@ -1,33 +1,18 @@ -using System.Text.Json.Serialization; +using FoundationaLLM.Common.Constants.Agents; +using System.Text.Json.Serialization; namespace FoundationaLLM.Common.Models.Orchestration { /// - /// Settings for an orchestration request. + /// Provides settings for a completion request of type . /// public class OrchestrationSettings { /// - /// The agent's LLM orchestrator type. - /// - [JsonPropertyName("orchestrator")] - public string? Orchestrator { get; set; } - - /// - /// AzureAICompletionParameters to override the behavior of the agent. - /// - [JsonPropertyName("agent_parameters")] - public Dictionary? AgentParameters { get; set; } - - /// - /// Options to override endpoint configuration (endpoint and key) used to - /// access a language model by the orchestrator. - /// - [JsonPropertyName("endpoint_configuration")] - public Dictionary? EndpointConfiguration { get; set; } - - /// - /// AzureAICompletionParameters to override the behavior of the language model as defined on the agent. + /// Dictionary with override values for the model parameters. + /// + /// For the list of supported keys, see . + /// /// [JsonPropertyName("model_parameters")] public Dictionary? ModelParameters { get; set; } diff --git a/src/dotnet/Common/Models/ResourceProviders/AIModel/AIModelBase.cs b/src/dotnet/Common/Models/ResourceProviders/AIModel/AIModelBase.cs new file mode 100644 index 0000000000..99b72bd71b --- /dev/null +++ b/src/dotnet/Common/Models/ResourceProviders/AIModel/AIModelBase.cs @@ -0,0 +1,48 @@ +using FoundationaLLM.Common.Constants.Agents; +using FoundationaLLM.Common.Constants.ResourceProviders; +using FoundationaLLM.Common.Models.ResourceProviders.Configuration; +using System.Text.Json.Serialization; + +namespace FoundationaLLM.Common.Models.ResourceProviders.AIModel +{ + /// + /// Provides a set of standard properties for all AIModel objects. + /// + [JsonPolymorphic(TypeDiscriminatorPropertyName = "type")] + [JsonDerivedType(typeof(EmbeddingAIModel), AIModelTypes.Embedding)] + [JsonDerivedType(typeof(CompletionAIModel), AIModelTypes.Completion)] + public class AIModelBase : ResourceBase + { + /// + [JsonIgnore] + public override string? Type { get; set; } + + /// + /// The object id of the object providing the configuration for the API endpoint used to interact with the model. + /// + [JsonPropertyName("endpoint_object_id")] + + public required string EndpointObjectId { get; set; } + + /// + /// The version of the AI model. + /// + [JsonPropertyName("version")] + public string? Version { get; set; } + + /// + /// The name of the deployment corresponding to the AI model. + /// + [JsonPropertyName("deployment_name")] + public string? DeploymentName { get; set; } + + /// + /// Dictionary with default values for the model parameters. + /// + /// For the list of supported keys, see . + /// + /// + [JsonPropertyName("model_parameters")] + public Dictionary ModelParameters { get; set; } = []; + } +} diff --git a/src/dotnet/Common/Models/ResourceProviders/AIModel/CompletionAIModel.cs b/src/dotnet/Common/Models/ResourceProviders/AIModel/CompletionAIModel.cs new file mode 100644 index 0000000000..4f08bd2c3e --- /dev/null +++ b/src/dotnet/Common/Models/ResourceProviders/AIModel/CompletionAIModel.cs @@ -0,0 +1,16 @@ +using FoundationaLLM.Common.Constants.ResourceProviders; + +namespace FoundationaLLM.Common.Models.ResourceProviders.AIModel +{ + /// + /// Provides the properties for AI models used for completions. + /// + public class CompletionAIModel : AIModelBase + { + /// + /// Creates a new instance of the AI model. + /// + public CompletionAIModel() => + Type = AIModelTypes.Completion; + } +} diff --git a/src/dotnet/Common/Models/ResourceProviders/AIModel/EmbeddingAIModel.cs b/src/dotnet/Common/Models/ResourceProviders/AIModel/EmbeddingAIModel.cs new file mode 100644 index 0000000000..38e660e20e --- /dev/null +++ b/src/dotnet/Common/Models/ResourceProviders/AIModel/EmbeddingAIModel.cs @@ -0,0 +1,16 @@ +using FoundationaLLM.Common.Constants.ResourceProviders; + +namespace FoundationaLLM.Common.Models.ResourceProviders.AIModel +{ + /// + /// Provides the properties for AI models used for embeddings. + /// + public class EmbeddingAIModel : AIModelBase + { + /// + /// Creates a new instance of the AI model. + /// + public EmbeddingAIModel() => + Type = AIModelTypes.Embedding; + } +} diff --git a/src/dotnet/Common/Models/ResourceProviders/Agent/AgentBase.cs b/src/dotnet/Common/Models/ResourceProviders/Agent/AgentBase.cs index 45c4e6630e..70b5eba3ea 100644 --- a/src/dotnet/Common/Models/ResourceProviders/Agent/AgentBase.cs +++ b/src/dotnet/Common/Models/ResourceProviders/Agent/AgentBase.cs @@ -1,5 +1,6 @@ using FoundationaLLM.Common.Exceptions; -using FoundationaLLM.Common.Models.Orchestration; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Common.Models.ResourceProviders.Prompt; using System.Text.Json.Serialization; namespace FoundationaLLM.Common.Models.ResourceProviders.Agent @@ -24,24 +25,31 @@ public class AgentBase : ResourceBase /// /// The agent's conversation history configuration. /// - [JsonPropertyName("conversation_history")] - public ConversationHistory? ConversationHistory { get; set; } + [JsonPropertyName("conversation_history_settings")] + public ConversationHistorySettings? ConversationHistorySettings { get; set; } /// /// The agent's Gatekeeper configuration. /// - [JsonPropertyName("gatekeeper")] - public Gatekeeper? Gatekeeper { get; set; } + [JsonPropertyName("gatekeeper_settings")] + public GatekeeperSettings? GatekeeperSettings { get; set; } /// /// Settings for the orchestration service. /// [JsonPropertyName("orchestration_settings")] - public OrchestrationSettings? OrchestrationSettings { get; set; } + public AgentOrchestrationSettings? OrchestrationSettings { get; set; } + /// - /// The agent's prompt. + /// The object identifier of the object providing the prompt for the agent. /// [JsonPropertyName("prompt_object_id")] public string? PromptObjectId { get; set; } + + /// + /// The object identifier of the object providing the AI model for the agent. + /// + [JsonPropertyName("ai_model_object_id")] + public string? AIModelObjectId { get; set; } /// /// Indicates whether the agent is long running and should use the polling pattern. @@ -65,7 +73,7 @@ public class AgentBase : ResourceBase /// /// Agent conversation history settings. /// - public class ConversationHistory + public class ConversationHistorySettings { /// /// Indicates whether the conversation history is enabled. @@ -82,7 +90,7 @@ public class ConversationHistory /// /// Agent Gatekeeper settings. /// - public class Gatekeeper + public class GatekeeperSettings { /// /// Indicates whether to abide by or override the system settings for the Gatekeeper. diff --git a/src/dotnet/Common/Models/ResourceProviders/Agent/AgentOrchestrationSettings.cs b/src/dotnet/Common/Models/ResourceProviders/Agent/AgentOrchestrationSettings.cs new file mode 100644 index 0000000000..653c2d7f0b --- /dev/null +++ b/src/dotnet/Common/Models/ResourceProviders/Agent/AgentOrchestrationSettings.cs @@ -0,0 +1,26 @@ +using FoundationaLLM.Common.Constants.Agents; +using System.Text.Json.Serialization; + +namespace FoundationaLLM.Common.Models.ResourceProviders.Agent +{ + /// + /// Provides agent-related orchestration settings. + /// + public class AgentOrchestrationSettings + { + /// + /// The agent's LLM orchestrator type. + /// + [JsonPropertyName("orchestrator")] + public string? Orchestrator { get; set; } + + /// + /// Dictionary with override values for the agent parameters. + /// + /// For the list of supported keys, see . + /// + /// + [JsonPropertyName("agent_parameters")] + public Dictionary? AgentParameters { get; set; } + } +} diff --git a/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpointCategory.cs b/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpointCategory.cs index 7583b83f3f..d9b5ed83f6 100644 --- a/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpointCategory.cs +++ b/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpointCategory.cs @@ -1,4 +1,4 @@ -namespace FoundationaLLM.Common.Models.ResourceProviders.Vectorization +namespace FoundationaLLM.Common.Models.ResourceProviders.Configuration { /// /// The category for api endpoint class. @@ -6,12 +6,17 @@ public enum APIEndpointCategory { /// - /// Endpoints related to orchestration. + /// Endpoints for internal orchestration services (e.g. LangChain API, SemanticKernel API). /// Orchestration, /// - /// Endpoints related to LLM. + /// Endpoints for external orchestration services. + /// + ExternalOrchestration, + + /// + /// Endpoints for Large Language Models. /// LLM, @@ -26,12 +31,12 @@ public enum APIEndpointCategory AzureAIDirect, /// - /// Endpoints for direct interactions with Microsoft Azure open AI services. + /// Endpoints for direct interactions with Microsoft Azure OpenAI services. /// AzureOpenAIDirect, /// - /// General endpoints. + /// General endpoints (internal APIs other than Orchestration, ExternalOrchestration, LLM, Gatekeeper, AzureAIDirect, or AzureOpenAIDirect. /// General } diff --git a/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpoint.cs b/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpointConfiguration.cs similarity index 55% rename from src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpoint.cs rename to src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpointConfiguration.cs index 1d31052a91..9bd5ac5160 100644 --- a/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpoint.cs +++ b/src/dotnet/Common/Models/ResourceProviders/Configuration/APIEndpointConfiguration.cs @@ -1,31 +1,33 @@ -using FoundationaLLM.Common.Constants.ResourceProviders; -using FoundationaLLM.Common.Models.ResourceProviders.Vectorization; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Constants.Configuration; +using FoundationaLLM.Common.Constants.ResourceProviders; using System.Text.Json.Serialization; namespace FoundationaLLM.Common.Models.ResourceProviders.Configuration { /// - /// Represents an api endpoint resource. + /// Provides the configuration for an API endpoint resource. /// - public class APIEndpoint : ResourceBase + public class APIEndpointConfiguration : ResourceBase { /// - /// Creates a new instance of . + /// Creates a new instance of . /// - public APIEndpoint() => - Type = ConfigurationTypes.APIEndpoint; + public APIEndpointConfiguration() => + Type = ConfigurationTypes.APIEndpointConfiguration; /// /// The api endpoint category. /// [JsonPropertyName("category")] + [JsonConverter(typeof(JsonStringEnumConverter))] public required APIEndpointCategory Category { get; set; } /// /// The type of authentication required for accessing the API. /// [JsonPropertyName("authentication_type")] - public required string AuthenticationType { get; set; } + public required AuthenticationTypes AuthenticationType { get; set; } /// /// The base URL of the API endpoint. @@ -37,43 +39,52 @@ public APIEndpoint() => /// A list of URL exceptions. /// [JsonPropertyName("url_exceptions")] - public List UrlExceptions { get; set; } = new List(); + public List UrlExceptions { get; set; } = []; /// - /// The API key used for authentication. + /// Dictionary with values used for authentication. + /// + /// For the list of supported keys, see . + /// /// - [JsonPropertyName("api_key")] - public string? APIKey { get; set; } + [JsonPropertyName("authentication_parameters")] + public Dictionary AuthenticationParameters { get; set; } = []; /// - /// The scope of the client. + /// The timeout duration in seconds for API calls. /// - [JsonPropertyName("scope")] - public string? Scope { get; set; } + [JsonPropertyName("timeout_seconds")] + public required int TimeoutSeconds { get; set; } /// - /// The api key configuration name. + /// The name of the retry strategy. /// - [JsonPropertyName("api_key_configuration_name")] - public string? APIKeyConfigurationName { get; set; } + [JsonPropertyName("retry_strategy_name")] + public required string RetryStrategyName { get; set; } /// - /// The api key header name. + /// The API endpoint provider. + /// + /// For a list of available API endpoint providers, see . + /// /// - [JsonPropertyName("api_key_header_name")] - public string? APIKeyHeaderName { get; set; } + [JsonPropertyName("provider")] + public string? Provider { get; set; } /// - /// The timeout duration in seconds for API calls. + /// The version to use when calling the API represented by the endpoint. /// - [JsonPropertyName("timeout_seconds")] - public required int TimeoutSeconds { get; set; } + [JsonPropertyName("api_version")] + public string? APIVersion { get; set; } /// - /// The name of the retry strategy. + /// Type of operation the endpoint is performing. + /// This value should be completions or chat. + /// Default value is chat. /// - [JsonPropertyName("retry_strategy_name")] - public required string RetryStrategyName { get; set; } + [JsonPropertyName("operation_type")] + public string? OperationType { get; set; } + } /// diff --git a/src/dotnet/Common/Services/Events/AzureEventGridEventService.cs b/src/dotnet/Common/Services/Events/AzureEventGridEventService.cs index 5ae292a535..fa00de74de 100644 --- a/src/dotnet/Common/Services/Events/AzureEventGridEventService.cs +++ b/src/dotnet/Common/Services/Events/AzureEventGridEventService.cs @@ -3,6 +3,7 @@ using Azure.Messaging.EventGrid.Namespaces; using FoundationaLLM.Common.Authentication; using FoundationaLLM.Common.Constants; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Configuration.Events; @@ -46,6 +47,10 @@ public class AzureEventGridEventService : IEventService { EventSetEventNamespaces.FoundationaLLM_ResourceProvider_Attachment, null + }, + { + EventSetEventNamespaces.FoundationaLLM_ResourceProvider_AIModel, + null } }; @@ -329,8 +334,8 @@ private void ValidateAPIKey(string? value) private EventGridClient? GetClient() => _settings.AuthenticationType switch { - AzureEventGridAuthenticationTypes.AzureIdentity => GetClientFromIdentity(), - AzureEventGridAuthenticationTypes.APIKey => GetClientFromAPIKey(), + AuthenticationTypes.AzureIdentity => GetClientFromIdentity(), + AuthenticationTypes.APIKey => GetClientFromAPIKey(), _ => throw new ConfigurationValueException($"The {_settings.AuthenticationType} authentication type is not supported by the Azure Event Grid events service.") }; diff --git a/src/dotnet/Common/Services/Storage/StorageServiceBase.cs b/src/dotnet/Common/Services/Storage/StorageServiceBase.cs index bba3dc2a90..72d65bb442 100644 --- a/src/dotnet/Common/Services/Storage/StorageServiceBase.cs +++ b/src/dotnet/Common/Services/Storage/StorageServiceBase.cs @@ -12,6 +12,7 @@ using System.Text; using System.Threading.Tasks; using FoundationaLLM.Common.Models.Configuration.Storage; +using FoundationaLLM.Common.Constants.Authentication; namespace FoundationaLLM.Common.Services.Storage { @@ -111,16 +112,16 @@ private void Initialize() { switch (_settings.AuthenticationType) { - case BlobStorageAuthenticationTypes.ConnectionString: + case AuthenticationTypes.ConnectionString: ValidateConnectionString(_settings.ConnectionString); CreateClientFromConnectionString(_settings.ConnectionString!); break; - case BlobStorageAuthenticationTypes.AccountKey: + case AuthenticationTypes.AccountKey: ValidateAccountName(_settings.AccountName); ValidateAccountKey(_settings.AccountKey); CreateClientFromAccountKey(_settings.AccountName!, _settings.AccountKey!); break; - case BlobStorageAuthenticationTypes.AzureIdentity: + case AuthenticationTypes.AzureIdentity: ValidateAccountName(_settings.AccountName); CreateClientFromIdentity(_settings.AccountName!); break; diff --git a/src/dotnet/Common/Settings/AzureAISearchAuthenticationTypes.cs b/src/dotnet/Common/Settings/AzureAISearchAuthenticationTypes.cs deleted file mode 100644 index 497cffc81c..0000000000 --- a/src/dotnet/Common/Settings/AzureAISearchAuthenticationTypes.cs +++ /dev/null @@ -1,24 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Text; -using System.Threading.Tasks; - -namespace FoundationaLLM.Common.Settings -{ - /// - /// Types of authentication for Azure AI Search. - /// - public enum AzureAISearchAuthenticationTypes - { - /// - /// Unknown authentication type. - /// - Unknown = -1, - - /// - /// Azure managed identity authentication type. - /// - AzureIdentity - } -} diff --git a/src/dotnet/Common/Settings/AzureOpenAIAuthenticationTypes.cs b/src/dotnet/Common/Settings/AzureOpenAIAuthenticationTypes.cs deleted file mode 100644 index a9fbe88e5b..0000000000 --- a/src/dotnet/Common/Settings/AzureOpenAIAuthenticationTypes.cs +++ /dev/null @@ -1,29 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Text; -using System.Threading.Tasks; - -namespace FoundationaLLM.Common.Settings -{ - /// - /// Types of authentication for Azure Open AI. - /// - public enum AzureOpenAIAuthenticationTypes - { - /// - /// Unknown authentication type. - /// - Unknown = -1, - - /// - /// Azure managed identity authentication type. - /// - AzureIdentity, - - /// - /// API key authentication type. - /// - APIKey - } -} diff --git a/src/dotnet/Configuration/Models/APIEndpointReference.cs b/src/dotnet/Configuration/Models/APIEndpointReference.cs index 1c21ec443c..f9aea9d4a1 100644 --- a/src/dotnet/Configuration/Models/APIEndpointReference.cs +++ b/src/dotnet/Configuration/Models/APIEndpointReference.cs @@ -18,7 +18,7 @@ public class APIEndpointReference : ResourceReference public Type ResourceType => Type switch { - ConfigurationTypes.APIEndpoint => typeof(APIEndpoint), + ConfigurationTypes.APIEndpointConfiguration => typeof(APIEndpointConfiguration), _ => throw new ResourceProviderException($"The resource type {Type} is not supported.") }; } diff --git a/src/dotnet/Configuration/Services/ConfigurationResourceProviderService.cs b/src/dotnet/Configuration/Services/ConfigurationResourceProviderService.cs index e114242ed3..742cae8d78 100644 --- a/src/dotnet/Configuration/Services/ConfigurationResourceProviderService.cs +++ b/src/dotnet/Configuration/Services/ConfigurationResourceProviderService.cs @@ -121,7 +121,7 @@ protected override async Task GetResourcesAsync(ResourcePath resourcePat resourcePath.ResourceTypeInstances[0].ResourceType switch { ConfigurationResourceTypeNames.AppConfigurations => await LoadAppConfigurationKeys(resourcePath.ResourceTypeInstances[0]), - ConfigurationResourceTypeNames.APIEndpoints => await LoadAPIEndpoints(resourcePath.ResourceTypeInstances[0]), + ConfigurationResourceTypeNames.APIEndpointConfigurations => await LoadAPIEndpoints(resourcePath.ResourceTypeInstances[0]), _ => throw new ResourceProviderException($"The resource type {resourcePath.ResourceTypeInstances[0].ResourceType} is not supported by the {_name} resource provider.", StatusCodes.Status400BadRequest) }; @@ -167,7 +167,7 @@ private async Task>> Loa return result; } - private async Task>> LoadAPIEndpoints(ResourceTypeInstance instance) + private async Task>> LoadAPIEndpoints(ResourceTypeInstance instance) { if (instance.ResourceId == null) { @@ -176,7 +176,7 @@ private async Task>> LoadAPIEndpoint .Where(apie => !apie.Deleted) .Select(apie => LoadAPIEndpoint(apie)))).ToList(); - return apiEndpoints.Select(service => new ResourceProviderGetResult() { Resource = service, Actions = [], Roles = [] }).ToList(); + return apiEndpoints.Select(service => new ResourceProviderGetResult() { Resource = service, Actions = [], Roles = [] }).ToList(); } else { @@ -187,17 +187,17 @@ private async Task>> LoadAPIEndpoint var apiEndpoint = await LoadAPIEndpoint(resourceReference); - return [new ResourceProviderGetResult() { Resource = apiEndpoint, Actions = [], Roles = [] }]; + return [new ResourceProviderGetResult() { Resource = apiEndpoint, Actions = [], Roles = [] }]; } } - private async Task LoadAPIEndpoint( + private async Task LoadAPIEndpoint( APIEndpointReference apiEndpointReference) { if (await _storageService.FileExistsAsync(_storageContainerName, apiEndpointReference.Filename, default)) { var fileContent = await _storageService.ReadFileAsync(_storageContainerName, apiEndpointReference.Filename, default); - return JsonSerializer.Deserialize( + return JsonSerializer.Deserialize( Encoding.UTF8.GetString(fileContent.ToArray()), _serializerSettings) ?? throw new ResourceProviderException($"Failed to load the api endpoint {apiEndpointReference.Name}.", @@ -215,7 +215,7 @@ protected override async Task UpsertResourceAsync(ResourcePath resourceP resourcePath.ResourceTypeInstances[0].ResourceType switch { ConfigurationResourceTypeNames.AppConfigurations => await UpdateAppConfigurationKey(resourcePath, serializedResource), - ConfigurationResourceTypeNames.APIEndpoints => await UpdateAPIEndpoints(resourcePath, serializedResource, userIdentity), + ConfigurationResourceTypeNames.APIEndpointConfigurations => await UpdateAPIEndpoints(resourcePath, serializedResource, userIdentity), _ => throw new ResourceProviderException($"The resource type {resourcePath.ResourceTypeInstances[0].ResourceType} is not supported by the {_name} resource provider.", StatusCodes.Status400BadRequest) }; @@ -225,7 +225,7 @@ protected override async Task DeleteResourceAsync(ResourcePath resourcePath, Uni { switch (resourcePath.ResourceTypeInstances.Last().ResourceType) { - case ConfigurationResourceTypeNames.APIEndpoints: + case ConfigurationResourceTypeNames.APIEndpointConfigurations: await DeleteAPIEndpoint(resourcePath.ResourceTypeInstances); break; case ConfigurationResourceTypeNames.AppConfigurations: @@ -286,7 +286,7 @@ await _appConfigurationService.SetConfigurationSettingAsync( private async Task UpdateAPIEndpoints(ResourcePath resourcePath, string serializedAPIEndpoint, UnifiedUserIdentity userIdentity) { - var apiEndpoint = JsonSerializer.Deserialize(serializedAPIEndpoint) + var apiEndpoint = JsonSerializer.Deserialize(serializedAPIEndpoint) ?? throw new ResourceProviderException("The object definition is invalid."); if (_apiEndpointReferences.TryGetValue(apiEndpoint.Name!, out var existingApiEndpointReference) @@ -331,7 +331,7 @@ await _storageService.WriteFileAsync( return new ResourceProviderUpsertResult { - ObjectId = (apiEndpoint as APIEndpoint)!.ObjectId + ObjectId = (apiEndpoint as APIEndpointConfiguration)!.ObjectId }; } diff --git a/src/dotnet/Core/Services/CoreService.cs b/src/dotnet/Core/Services/CoreService.cs index 89cb8bea9f..d215986678 100644 --- a/src/dotnet/Core/Services/CoreService.cs +++ b/src/dotnet/Core/Services/CoreService.cs @@ -205,14 +205,14 @@ private async Task ProcessGatekeeperOptions(Compl var agentBase = await agentResourceProvider.GetResource($"/{AgentResourceTypeNames.Agents}/{completionRequest.AgentName}", _callContext.CurrentUserIdentity ?? throw new InvalidOperationException("Failed to retrieve the identity of the signed in user when retrieving the agent settings.")); - if (agentBase?.Gatekeeper?.UseSystemSetting == false) + if (agentBase?.GatekeeperSettings?.UseSystemSetting == false) { // Agent does not want to use system settings, however it does not have any Gatekeeper options either // Consequently, a request to bypass Gatekeeper will be returned. - if (agentBase!.Gatekeeper!.Options == null || agentBase.Gatekeeper.Options.Length == 0) + if (agentBase!.GatekeeperSettings!.Options == null || agentBase.GatekeeperSettings.Options.Length == 0) return AgentGatekeeperOverrideOption.MustBypass; - completionRequest.GatekeeperOptions = agentBase.Gatekeeper.Options; + completionRequest.GatekeeperOptions = agentBase.GatekeeperSettings.Options; return AgentGatekeeperOverrideOption.MustCall; } diff --git a/src/dotnet/CoreAPI/Program.cs b/src/dotnet/CoreAPI/Program.cs index 3ea7a81df4..d5a1b9c66a 100644 --- a/src/dotnet/CoreAPI/Program.cs +++ b/src/dotnet/CoreAPI/Program.cs @@ -60,6 +60,8 @@ public static void Main(string[] args) options.Select(AppConfigurationKeyFilters.FoundationaLLM_Agent); options.Select(AppConfigurationKeyFilters.FoundationaLLM_Events); options.Select(AppConfigurationKeyFilters.FoundationaLLM_Attachment); + options.Select(AppConfigurationKeyFilters.FoundationaLLM_AIModel); + }); if (builder.Environment.IsDevelopment()) builder.Configuration.AddJsonFile("appsettings.development.json", true, true); diff --git a/src/dotnet/Gateway/Models/EmbeddingModelDeploymentContext.cs b/src/dotnet/Gateway/Models/EmbeddingModelDeploymentContext.cs index 807cc41de8..45c218d106 100644 --- a/src/dotnet/Gateway/Models/EmbeddingModelDeploymentContext.cs +++ b/src/dotnet/Gateway/Models/EmbeddingModelDeploymentContext.cs @@ -1,4 +1,5 @@ -using FoundationaLLM.Common.Interfaces; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Azure; using FoundationaLLM.Common.Models.Gateway; using FoundationaLLM.Common.Models.Vectorization; @@ -31,7 +32,7 @@ public class EmbeddingModelDeploymentContext( private readonly ITextEmbeddingService _textEmbeddingService = new SemanticKernelTextEmbeddingService( Options.Create(new SemanticKernelTextEmbeddingServiceSettings { - AuthenticationType = AzureOpenAIAuthenticationTypes.AzureIdentity, + AuthenticationType = AuthenticationTypes.AzureIdentity, Endpoint = deployment.AccountEndpoint, DeploymentName = deployment.Name }), diff --git a/src/dotnet/Management/Management.csproj b/src/dotnet/Management/Management.csproj index 4227a5af5a..e1b1e83e3b 100644 --- a/src/dotnet/Management/Management.csproj +++ b/src/dotnet/Management/Management.csproj @@ -25,6 +25,7 @@ + diff --git a/src/dotnet/ManagementAPI/Program.cs b/src/dotnet/ManagementAPI/Program.cs index aacc9e4fd3..5997eb3d08 100644 --- a/src/dotnet/ManagementAPI/Program.cs +++ b/src/dotnet/ManagementAPI/Program.cs @@ -13,7 +13,6 @@ using FoundationaLLM.Common.Settings; using FoundationaLLM.Common.Validation; using FoundationaLLM.Management.Models.Configuration; -using FoundationaLLM.Vectorization.Client; using FoundationaLLM.Vectorization.Interfaces; using FoundationaLLM.Vectorization.Models.Configuration; using FoundationaLLM.Vectorization.Services.RequestProcessors; @@ -59,7 +58,8 @@ public static async Task Main(string[] args) options.Select(AppConfigurationKeyFilters.FoundationaLLM_DataSource); //resource provider settings options.Select(AppConfigurationKeyFilters.FoundationaLLM_Events); options.Select(AppConfigurationKeyFilters.FoundationaLLM_Configuration); - options.Select(AppConfigurationKeyFilters.FoundationaLLM_Attachment); + options.Select(AppConfigurationKeyFilters.FoundationaLLM_Attachment); + options.Select(AppConfigurationKeyFilters.FoundationaLLM_AIModel); }); if (builder.Environment.IsDevelopment()) @@ -117,6 +117,7 @@ public static async Task Main(string[] args) builder.AddPromptResourceProvider(); builder.AddDataSourceResourceProvider(); builder.AddAttachmentResourceProvider(); + builder.AddAIModelResourceProvider(); // Add authentication configuration. var e2ETestEnvironmentValue = Environment.GetEnvironmentVariable(EnvironmentVariables.FoundationaLLM_Environment) ?? string.Empty; diff --git a/src/dotnet/ManagementClient/Clients/Resources/ConfigurationManagementClient.cs b/src/dotnet/ManagementClient/Clients/Resources/ConfigurationManagementClient.cs index 2a84ba3972..d220f92eee 100644 --- a/src/dotnet/ManagementClient/Clients/Resources/ConfigurationManagementClient.cs +++ b/src/dotnet/ManagementClient/Clients/Resources/ConfigurationManagementClient.cs @@ -22,18 +22,18 @@ await managementRestClient.Resources.GetResourcesAsync - public async Task>> GetExternalOrchestrationServicesAsync() => - await managementRestClient.Resources.GetResourcesAsync>>( + public async Task>> GetExternalOrchestrationServicesAsync() => + await managementRestClient.Resources.GetResourcesAsync>>( ResourceProviderNames.FoundationaLLM_Configuration, - ConfigurationResourceTypeNames.APIEndpoints + ConfigurationResourceTypeNames.APIEndpointConfigurations ); /// - public async Task> GetExternalOrchestrationServiceAsync(string externalOrchestrationServiceName) + public async Task> GetExternalOrchestrationServiceAsync(string externalOrchestrationServiceName) { - var result = await managementRestClient.Resources.GetResourcesAsync>>( + var result = await managementRestClient.Resources.GetResourcesAsync>>( ResourceProviderNames.FoundationaLLM_Configuration, - $"{ConfigurationResourceTypeNames.APIEndpoints}/{externalOrchestrationServiceName}" + $"{ConfigurationResourceTypeNames.APIEndpointConfigurations}/{externalOrchestrationServiceName}" ); if (result == null || result.Count == 0) diff --git a/src/dotnet/ManagementClient/Interfaces/IConfigurationManagementClient.cs b/src/dotnet/ManagementClient/Interfaces/IConfigurationManagementClient.cs index 45161782c5..9978235e5f 100644 --- a/src/dotnet/ManagementClient/Interfaces/IConfigurationManagementClient.cs +++ b/src/dotnet/ManagementClient/Interfaces/IConfigurationManagementClient.cs @@ -28,7 +28,7 @@ public interface IConfigurationManagementClient /// Retrieves all external orchestration services. /// /// - Task>> GetExternalOrchestrationServicesAsync(); + Task>> GetExternalOrchestrationServicesAsync(); /// /// Returns a specific external orchestration service by name. @@ -36,7 +36,7 @@ public interface IConfigurationManagementClient /// The name of the external orchestration /// service to retrieve. /// - Task> GetExternalOrchestrationServiceAsync(string externalOrchestrationServiceName); + Task> GetExternalOrchestrationServiceAsync(string externalOrchestrationServiceName); /// /// Upserts an app configuration value. If the value does not exist, it will be created. diff --git a/src/dotnet/Orchestration/Orchestration.csproj b/src/dotnet/Orchestration/Orchestration.csproj index cb9dca145b..67f4a93884 100644 --- a/src/dotnet/Orchestration/Orchestration.csproj +++ b/src/dotnet/Orchestration/Orchestration.csproj @@ -23,6 +23,7 @@ + diff --git a/src/dotnet/Orchestration/Orchestration/KnowledgeManagementOrchestration.cs b/src/dotnet/Orchestration/Orchestration/KnowledgeManagementOrchestration.cs index 99f87b2433..fd1c464d76 100644 --- a/src/dotnet/Orchestration/Orchestration/KnowledgeManagementOrchestration.cs +++ b/src/dotnet/Orchestration/Orchestration/KnowledgeManagementOrchestration.cs @@ -18,6 +18,7 @@ namespace FoundationaLLM.Orchestration.Core.Orchestration /// Constructor for default agent. /// /// The agent. + /// A dictionary of objects retrieved from various object ids related to the agent. For more details see . /// The call context of the request being handled. /// /// The logger used for logging. @@ -25,15 +26,17 @@ namespace FoundationaLLM.Orchestration.Core.Orchestration /// Inidicates that access was denied to all underlying data sources. public class KnowledgeManagementOrchestration( KnowledgeManagementAgent agent, + Dictionary explodedObjects, ICallContext callContext, ILLMOrchestrationService orchestrationService, ILogger logger, Dictionary resourceProviderServices, bool dataSourceAccessDenied) : OrchestrationBase(orchestrationService) { + private readonly KnowledgeManagementAgent _agent = agent; + private readonly Dictionary _explodedObjects = explodedObjects; private readonly ICallContext _callContext = callContext; private readonly ILogger _logger = logger; - private readonly KnowledgeManagementAgent _agent = agent; private readonly Dictionary _resourceProviderServices = resourceProviderServices; private readonly bool _dataSourceAccessDenied = dataSourceAccessDenied; @@ -61,10 +64,10 @@ public override async Task GetCompletion(string instanceId, new LLMCompletionRequest { UserPrompt = completionRequest.UserPrompt!, - Agent = _agent, MessageHistory = completionRequest.MessageHistory, Attachments = completionRequest.Attachments == null ? [] : await GetAttachmentPaths(completionRequest.Attachments), - Settings = completionRequest.Settings + Agent = _agent, + Objects = _explodedObjects }); if (result.Citations != null) diff --git a/src/dotnet/Orchestration/Orchestration/OrchestrationBase.cs b/src/dotnet/Orchestration/Orchestration/OrchestrationBase.cs index 1993dcc512..25d1a6d015 100644 --- a/src/dotnet/Orchestration/Orchestration/OrchestrationBase.cs +++ b/src/dotnet/Orchestration/Orchestration/OrchestrationBase.cs @@ -8,7 +8,7 @@ namespace FoundationaLLM.Orchestration.Core.Orchestration /// Base class for an orchestration involving a FoundationaLLM agent. /// /// - /// Constructor for the AgentBase class. + /// Constructor for the OrchestrationBase class. /// /// public class OrchestrationBase(ILLMOrchestrationService orchestrationService) diff --git a/src/dotnet/Orchestration/Orchestration/OrchestrationBuilder.cs b/src/dotnet/Orchestration/Orchestration/OrchestrationBuilder.cs index cee796b53a..ad82504f9a 100644 --- a/src/dotnet/Orchestration/Orchestration/OrchestrationBuilder.cs +++ b/src/dotnet/Orchestration/Orchestration/OrchestrationBuilder.cs @@ -1,10 +1,14 @@ using FoundationaLLM.Common.Constants; +using FoundationaLLM.Common.Constants.Agents; using FoundationaLLM.Common.Constants.ResourceProviders; using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Extensions; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Authentication; +using FoundationaLLM.Common.Models.Orchestration; using FoundationaLLM.Common.Models.ResourceProviders.Agent; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Common.Models.ResourceProviders.Configuration; using FoundationaLLM.Common.Models.ResourceProviders.DataSource; using FoundationaLLM.Common.Models.ResourceProviders.Prompt; using FoundationaLLM.Common.Models.ResourceProviders.Vectorization; @@ -22,9 +26,14 @@ public class OrchestrationBuilder { /// /// Builds the orchestration based on the user prompt, the session id, and the call context. + /// + /// Note that the agent name in can be different from the agent name in . + /// This happens when the original completion request results in the need to bring in additional agents to the conversation. + /// /// /// The FoundationaLLM instance ID. /// The unique name of the agent for which the orchestration is built. + /// The request for which the orchestration is built. /// The call context of the request being handled. /// The used to retrieve app settings from configuration. /// A dictionary of resource providers hashed by resource provider name. @@ -36,6 +45,7 @@ public class OrchestrationBuilder public static async Task Build( string instanceId, string agentName, + CompletionRequest originalRequest, ICallContext callContext, IConfiguration configuration, Dictionary resourceProviderServices, @@ -45,7 +55,13 @@ public class OrchestrationBuilder { var logger = loggerFactory.CreateLogger(); - var result = await LoadAgent(agentName, resourceProviderServices, callContext.CurrentUserIdentity!, logger); + var result = await LoadAgent( + agentName, + originalRequest.Settings?.ModelParameters, + resourceProviderServices, + callContext.CurrentUserIdentity!, + logger); + if (result.Agent == null) return null; if (result.Agent.AgentType == typeof(KnowledgeManagementAgent)) @@ -58,6 +74,7 @@ public class OrchestrationBuilder var kmOrchestration = new KnowledgeManagementOrchestration( (KnowledgeManagementAgent)result.Agent, + result.ExplodedObjects ?? [], callContext, orchestrationService, loggerFactory.CreateLogger(), @@ -70,8 +87,9 @@ public class OrchestrationBuilder return null; } - private static async Task<(AgentBase? Agent, bool DataSourceAccessDenied)> LoadAgent( + private static async Task<(AgentBase? Agent, Dictionary? ExplodedObjects, bool DataSourceAccessDenied)> LoadAgent( string? agentName, + Dictionary? modelParameterOverrides, Dictionary resourceProviderServices, UnifiedUserIdentity currentUserIdentity, ILogger logger) @@ -87,19 +105,40 @@ public class OrchestrationBuilder throw new OrchestrationException($"The resource provider {ResourceProviderNames.FoundationaLLM_Vectorization} was not loaded."); if (!resourceProviderServices.TryGetValue(ResourceProviderNames.FoundationaLLM_DataSource, out var dataSourceResourceProvider)) throw new OrchestrationException($"The resource provider {ResourceProviderNames.FoundationaLLM_DataSource} was not loaded."); + if (!resourceProviderServices.TryGetValue(ResourceProviderNames.FoundationaLLM_AIModel, out var aiModelResourceProvider)) + throw new OrchestrationException($"The resource provider {ResourceProviderNames.FoundationaLLM_AIModel} was not loaded."); + if (!resourceProviderServices.TryGetValue(ResourceProviderNames.FoundationaLLM_Configuration, out var configurationResourceProvider)) + throw new OrchestrationException($"The resource provider {ResourceProviderNames.FoundationaLLM_Configuration} was not loaded."); + + var explodedObjects = new Dictionary(); var agentBase = await agentResourceProvider.GetResource( $"/{AgentResourceTypeNames.Agents}/{agentName}", currentUserIdentity); - if (agentBase.OrchestrationSettings!.AgentParameters == null) - agentBase.OrchestrationSettings.AgentParameters = []; - var prompt = await promptResourceProvider.GetResource( agentBase.PromptObjectId!, currentUserIdentity); + var aiModel = await aiModelResourceProvider.GetResource( + agentBase.AIModelObjectId!, + currentUserIdentity); + var apiEndpointConfiguration = await configurationResourceProvider.GetResource( + aiModel.EndpointObjectId!, + currentUserIdentity); + + // Merge the model parameter overrides with the existing model parameter values from the AI model. + if (modelParameterOverrides != null) + { + // Allowing the override only for the keys that are supported. + foreach (var key in modelParameterOverrides.Keys.Where(k => ModelParametersKeys.All.Contains(k))) + { + aiModel.ModelParameters[key] = modelParameterOverrides[key]; + } + } - agentBase.OrchestrationSettings.AgentParameters[agentBase.PromptObjectId!] = prompt; + explodedObjects[agentBase.PromptObjectId!] = prompt; + explodedObjects[agentBase.AIModelObjectId!] = aiModel; + explodedObjects[aiModel.EndpointObjectId!] = apiEndpointConfiguration; var allAgents = await agentResourceProvider.GetResources(currentUserIdentity); var allAgentsDescriptions = allAgents @@ -110,7 +149,7 @@ public class OrchestrationBuilder a.Description }) .ToDictionary(x => x.Name, x => x.Description); - agentBase.OrchestrationSettings.AgentParameters["AllAgents"] = allAgentsDescriptions; + explodedObjects[CompletionRequestObjectsKeys.AllAgents] = allAgentsDescriptions; if (agentBase is KnowledgeManagementAgent kmAgent) { @@ -126,12 +165,12 @@ public class OrchestrationBuilder currentUserIdentity); if (dataSource == null) - return (null, false); + return (null, null, false); } catch (ResourceProviderException ex) when (ex.StatusCode == (int)HttpStatusCode.Forbidden) { // Access is denied to the underlying data source. - return (agentBase, true); + return (agentBase, null, true); } } @@ -141,7 +180,7 @@ public class OrchestrationBuilder indexingProfileName, currentUserIdentity); - kmAgent.OrchestrationSettings!.AgentParameters![indexingProfileName] = indexingProfile; + explodedObjects[indexingProfileName] = indexingProfile; } if (!string.IsNullOrWhiteSpace(kmAgent.Vectorization.TextEmbeddingProfileObjectId)) @@ -150,12 +189,12 @@ public class OrchestrationBuilder kmAgent.Vectorization.TextEmbeddingProfileObjectId, currentUserIdentity); - kmAgent.OrchestrationSettings!.AgentParameters![kmAgent.Vectorization.TextEmbeddingProfileObjectId!] = textEmbeddingProfile; + explodedObjects[kmAgent.Vectorization.TextEmbeddingProfileObjectId!] = textEmbeddingProfile; } } } - return (agentBase, false); + return (agentBase, explodedObjects, false); } } } diff --git a/src/dotnet/Orchestration/Services/AzureAIDirectService.cs b/src/dotnet/Orchestration/Services/AzureAIDirectService.cs index 205edf58a9..ce2bac1b4f 100644 --- a/src/dotnet/Orchestration/Services/AzureAIDirectService.cs +++ b/src/dotnet/Orchestration/Services/AzureAIDirectService.cs @@ -1,17 +1,18 @@ using FoundationaLLM.Common.Constants; using FoundationaLLM.Common.Constants.Agents; -using FoundationaLLM.Common.Constants.ResourceProviders; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Extensions; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Infrastructure; using FoundationaLLM.Common.Models.Orchestration; using FoundationaLLM.Common.Models.Orchestration.Direct; -using FoundationaLLM.Common.Models.ResourceProviders.Prompt; using FoundationaLLM.Common.Settings; using FoundationaLLM.Orchestration.Core.Interfaces; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.Logging; +using Microsoft.Net.Http.Headers; +using System.Net.Http.Headers; using System.Text; using System.Text.Json; @@ -55,53 +56,36 @@ await Task.FromResult(new ServiceStatusInfo /// public async Task GetCompletion(string instanceId, LLMCompletionRequest request) { - var agent = request.Agent - ?? throw new Exception("Agent cannot be null."); - - var endpointConfiguration = (agent.OrchestrationSettings?.EndpointConfiguration) - ?? throw new Exception("Endpoint Configuration must be provided."); - - var endpointSettings = GetEndpointSettings(endpointConfiguration); + request.Validate(); SystemCompletionMessage? systemPrompt = null; CompletionMessage? assistantPrompt = null; - if (!string.IsNullOrWhiteSpace(agent.PromptObjectId)) + + // We are adding an empty assistant prompt and setting the system prompt to the user role to support + // some models (like Mistral) that require user/assistant prompts and not system prompts. + var inputStrings = new List() { - if (!_resourceProviderServices.TryGetValue(ResourceProviderNames.FoundationaLLM_Prompt, out var promptResourceProvider)) - throw new ResourceProviderException($"The resource provider {ResourceProviderNames.FoundationaLLM_Prompt} was not loaded."); - - var prompt = await promptResourceProvider.GetResource(agent.PromptObjectId, _callContext.CurrentUserIdentity!) as MultipartPrompt; - - // We are adding an empty assistant prompt and setting the system prompt to the user role to support - // some models (like Mistral) that require user/assistant prompts and not system prompts. - systemPrompt = new SystemCompletionMessage + new SystemCompletionMessage { Role = InputMessageRoles.User, - Content = prompt?.Prefix ?? string.Empty - }; - assistantPrompt = new CompletionMessage + Content = request.Prompt.Prefix ?? string.Empty + }, + new CompletionMessage { Role = InputMessageRoles.Assistant, Content = string.Empty - }; - } + } + }; - var inputStrings = new List(); - // Add system prompt, if exists. - if (systemPrompt != null) - { - inputStrings.Add(systemPrompt); - inputStrings.Add(assistantPrompt!); - } // Add conversation history. - if (agent.ConversationHistory?.Enabled == true && request.MessageHistory != null) + if (request.Agent.ConversationHistorySettings?.Enabled == true && request.MessageHistory != null) { // The message history needs to be in a continuous order of user and assistant messages. // If the MaxHistory value is odd, add one to the number of messages to take to ensure proper pairing. - if (agent.ConversationHistory.MaxHistory % 2 != 0) - agent.ConversationHistory.MaxHistory++; + if (request.Agent.ConversationHistorySettings.MaxHistory % 2 != 0) + request.Agent.ConversationHistorySettings.MaxHistory++; - var messageHistoryItems = request.MessageHistory?.TakeLast(agent.ConversationHistory.MaxHistory); + var messageHistoryItems = request.MessageHistory?.TakeLast(request.Agent.ConversationHistorySettings.MaxHistory); foreach(var item in messageHistoryItems!) { @@ -116,68 +100,86 @@ public async Task GetCompletion(string instanceId, LLMCom var userPrompt = new UserCompletionMessage { Content = request.UserPrompt }; inputStrings.Add(userPrompt); - if (!string.IsNullOrWhiteSpace(endpointSettings.Endpoint) && !string.IsNullOrWhiteSpace(endpointSettings.APIKey)) + var endpointConfiguration = request.AIModelEndpointConfiguration; + var apiKey = string.Empty; + var apiKeyHeaderName = string.Empty; + var apiKeyPrefix = string.Empty; + + if (endpointConfiguration.AuthenticationType == AuthenticationTypes.APIKey) + { + if (!endpointConfiguration.AuthenticationParameters.TryGetValue(AuthenticationParametersKeys.APIKeyConfigurationName, out var apiKeyKeyName)) + throw new OrchestrationException($"The {AuthenticationParametersKeys.APIKeyConfigurationName} key is missing from the AI model enpoint's authentication parameters dictionary."); + + apiKey = _configuration.GetValue(apiKeyKeyName?.ToString()!)!; + } + + // The expected value for the header name is "Authorization". + if (!endpointConfiguration.AuthenticationParameters.TryGetValue(AuthenticationParametersKeys.APIKeyHeaderName, out var apiKeyHeaderNameObject)) + throw new OrchestrationException($"The {AuthenticationParametersKeys.APIKeyHeaderName} key is missing from the AI model enpoint's authentication parameters dictionary."); + apiKeyHeaderName = apiKeyHeaderNameObject.ToString(); + + // The optional expected value for the API key prefix is "Bearer". + if (endpointConfiguration.AuthenticationParameters.TryGetValue(AuthenticationParametersKeys.APIKeyPrefix, out var apiKeyPrefixObject)) + apiKeyPrefix = apiKeyPrefixObject.ToString(); + + if (!string.IsNullOrWhiteSpace(endpointConfiguration.Url) + && !string.IsNullOrWhiteSpace(apiKey) + && !string.IsNullOrWhiteSpace(apiKeyHeaderName)) { var client = _httpClientFactoryService.CreateClient(HttpClients.AzureAIDirect); - if (endpointSettings.AuthenticationType == "key" && !string.IsNullOrWhiteSpace(endpointSettings.APIKey)) + + if (apiKeyHeaderName == HeaderNames.Authorization) { - client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue( - "Bearer", endpointSettings.APIKey + client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue( + apiKeyPrefix ?? string.Empty, apiKey ); } + else + { + client.DefaultRequestHeaders.Add( + apiKeyHeaderName, + string.IsNullOrWhiteSpace(apiKeyPrefix) + ? apiKey + : $"{apiKeyPrefix} {apiKey}"); + } - client.BaseAddress = new Uri(endpointSettings.Endpoint); - - var modelParameters = agent.OrchestrationSettings?.ModelParameters; - var modelOverrides = request.Settings?.ModelParameters; - - AzureAICompletionRequest azureAiCompletionRequest; + client.BaseAddress = new Uri(endpointConfiguration.Url!); - if (modelParameters != null) - { - azureAiCompletionRequest = new() - { - InputData = new() - { - InputString = [.. inputStrings], - Parameters = modelParameters.ToObject(modelOverrides) - } - }; + var modelParameters = request.AIModel.ModelParameters; - if (modelOverrides != null && modelOverrides.ContainsKey(ModelParameterKeys.DeploymentName)) + AzureAICompletionRequest azureAiCompletionRequest = new() + { + InputData = new() { - modelParameters[ModelParameterKeys.DeploymentName] = modelOverrides[ModelParameterKeys.DeploymentName]; + InputString = [.. inputStrings], + Parameters = modelParameters.ToObject() } + }; - var body = JsonSerializer.Serialize(azureAiCompletionRequest, _jsonSerializerOptions); - var content = new StringContent(body, Encoding.UTF8, "application/json"); - if (modelParameters.TryGetValue(ModelParameterKeys.DeploymentName, out var deployment)) - { - content.Headers.Add("azureml-model-deployment", deployment.ToString()); - } + var body = JsonSerializer.Serialize(azureAiCompletionRequest, _jsonSerializerOptions); + var content = new StringContent(body, Encoding.UTF8, "application/json"); - var responseMessage = await client.PostAsync("", content); - var responseContent = await responseMessage.Content.ReadAsStringAsync(); + var responseMessage = await client.PostAsync("", content); + var responseContent = await responseMessage.Content.ReadAsStringAsync(); - if (responseMessage.IsSuccessStatusCode) - { - var completionResponse = JsonSerializer.Deserialize(responseContent); - - return new LLMCompletionResponse - { - Completion = completionResponse!.Output, - UserPrompt = request.UserPrompt, - FullPrompt = body, - PromptTemplate = systemPrompt?.Content, - AgentName = agent.Name, - PromptTokens = 0, - CompletionTokens = 0 - }; - } + if (responseMessage.IsSuccessStatusCode) + { + var completionResponse = JsonSerializer.Deserialize(responseContent); - _logger.LogWarning("The AzureAIDirect orchestration service returned status code {StatusCode}: {ResponseContent}", - responseMessage.StatusCode, responseContent); + return new LLMCompletionResponse + { + Completion = completionResponse!.Output, + UserPrompt = request.UserPrompt, + FullPrompt = body, + PromptTemplate = systemPrompt?.Content, + AgentName = request.Agent.Name, + PromptTokens = 0, + CompletionTokens = 0 + }; } + + _logger.LogWarning("The AzureAIDirect orchestration service returned status code {StatusCode}: {ResponseContent}", + responseMessage.StatusCode, responseContent); } return new LLMCompletionResponse @@ -185,43 +187,10 @@ public async Task GetCompletion(string instanceId, LLMCom Completion = "A problem on my side prevented me from responding.", UserPrompt = request.UserPrompt, PromptTemplate = systemPrompt?.Content, - AgentName = agent.Name, + AgentName = request.Agent.Name, PromptTokens = 0, CompletionTokens = 0 }; } - - /// - /// Extracts endpoint configuration values from a dictionary and writes them into a - /// object. - /// - /// Dictionary containing orchestration endpoint configuration values. - /// Returns a object containing the endpoint configuration. - /// - private EndpointSettings GetEndpointSettings(Dictionary endpointConfiguration) - { - if (!endpointConfiguration.TryGetValue(EndpointConfigurationKeys.Endpoint, out var endpointKeyName)) - throw new Exception("An endpoint value must be passed in via an Azure App Config key name."); - - var endpoint = _configuration.GetValue(endpointKeyName?.ToString()!); - - var authenticationType = endpointConfiguration.GetValueOrDefault(EndpointConfigurationKeys.AuthenticationType, "key").ToString(); - var apiKey = string.Empty; - - if (authenticationType == "key") - { - if (!endpointConfiguration.TryGetValue(EndpointConfigurationKeys.APIKey, out var apiKeyKeyName)) - throw new Exception("An API key must be passed in via an Azure App Config key name."); - - apiKey = _configuration.GetValue(apiKeyKeyName?.ToString()!)!; - } - - return new EndpointSettings - { - Endpoint = endpoint!, - APIKey = apiKey!, - AuthenticationType = authenticationType! - }; - } } } diff --git a/src/dotnet/Orchestration/Services/AzureOpenAIDirectService.cs b/src/dotnet/Orchestration/Services/AzureOpenAIDirectService.cs index 74fd8c4456..929fe0ee10 100644 --- a/src/dotnet/Orchestration/Services/AzureOpenAIDirectService.cs +++ b/src/dotnet/Orchestration/Services/AzureOpenAIDirectService.cs @@ -1,7 +1,7 @@ -using System.Text; -using System.Text.Json; +using AngleSharp.Dom.Events; using FoundationaLLM.Common.Constants; using FoundationaLLM.Common.Constants.Agents; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Constants.ResourceProviders; using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Extensions; @@ -14,12 +14,15 @@ using FoundationaLLM.Orchestration.Core.Interfaces; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.Logging; +using System.Text; +using System.Text.Json; namespace FoundationaLLM.Orchestration.Core.Services { /// /// The Azure OpenAI direct orchestration service. /// + /// The providing details about the call context. /// The logger used for logging. /// The used to retrieve app settings from configuration. /// The HTTP client factory service. @@ -54,45 +57,30 @@ await Task.FromResult(new ServiceStatusInfo /// public async Task GetCompletion(string instanceId, LLMCompletionRequest request) { - var agent = request.Agent - ?? throw new Exception("Agent cannot be null."); - - var endpointConfiguration = (agent.OrchestrationSettings?.EndpointConfiguration) - ?? throw new Exception("Endpoint Configuration must be provided."); - - var endpointSettings = GetEndpointSettings(endpointConfiguration); - + request.Validate(); + var endpointConfiguration = request.AIModelEndpointConfiguration; var inputStrings = new List(); SystemCompletionMessage? systemPrompt = null; - if (endpointSettings.OperationType == OperationTypes.Chat) + if (!string.IsNullOrWhiteSpace(endpointConfiguration.OperationType) + && endpointConfiguration.OperationType == OperationTypes.Chat) { - if (!string.IsNullOrWhiteSpace(agent.PromptObjectId)) + inputStrings.Add(new SystemCompletionMessage { - if (!_resourceProviderServices.TryGetValue(ResourceProviderNames.FoundationaLLM_Prompt, out var promptResourceProvider)) - throw new ResourceProviderException($"The resource provider {ResourceProviderNames.FoundationaLLM_Prompt} was not loaded."); - - var prompt = await promptResourceProvider.GetResource(agent.PromptObjectId, _callContext.CurrentUserIdentity!) as MultipartPrompt; + Role = InputMessageRoles.System, + Content = request.Prompt.Prefix ?? string.Empty + }); - systemPrompt = new SystemCompletionMessage - { - Role = InputMessageRoles.System, - Content = prompt?.Prefix ?? string.Empty - }; - } - - // Add system prompt, if exists. - if (systemPrompt != null) inputStrings.Add(systemPrompt); // Add conversation history. - if (agent.ConversationHistory?.Enabled == true && request.MessageHistory != null) + if (request.Agent.ConversationHistorySettings?.Enabled == true && request.MessageHistory != null) { // The message history needs to be in a continuous order of user and assistant messages. // If the MaxHistory value is odd, add one to the number of messages to take to ensure proper pairing. - if (agent.ConversationHistory.MaxHistory % 2 != 0) - agent.ConversationHistory.MaxHistory++; + if (request.Agent.ConversationHistorySettings.MaxHistory % 2 != 0) + request.Agent.ConversationHistorySettings.MaxHistory++; - var messageHistoryItems = request.MessageHistory?.TakeLast(agent.ConversationHistory.MaxHistory); + var messageHistoryItems = request.MessageHistory?.TakeLast(request.Agent.ConversationHistorySettings.MaxHistory); foreach (var item in messageHistoryItems!) { inputStrings.Add(new CompletionMessage @@ -107,68 +95,78 @@ public async Task GetCompletion(string instanceId, LLMCom inputStrings.Add(userPrompt); } - if (!string.IsNullOrWhiteSpace(endpointSettings.Endpoint) && !string.IsNullOrWhiteSpace(endpointSettings.APIKey)) + var apiKey = string.Empty; + var apiKeyHeaderName = string.Empty; + + if (endpointConfiguration.AuthenticationType == AuthenticationTypes.APIKey) + { + if (!endpointConfiguration.AuthenticationParameters.TryGetValue( + AuthenticationParametersKeys.APIKeyConfigurationName, out var apiKeyKeyName)) + throw new OrchestrationException($"The {AuthenticationParametersKeys.APIKeyConfigurationName} key is missing from the AI model enpoint's authentication parameters dictionary."); + + apiKey = _configuration.GetValue(apiKeyKeyName?.ToString()!)!; + } + + // The expected value for the header name is "api-key". + if (!endpointConfiguration.AuthenticationParameters.TryGetValue(AuthenticationParametersKeys.APIKeyHeaderName, out var apiKeyHeaderNameObject)) + throw new OrchestrationException($"The {AuthenticationParametersKeys.APIKeyHeaderName} key is missing from the AI model enpoint's authentication parameters dictionary."); + apiKeyHeaderName = apiKeyHeaderNameObject.ToString(); + + if (!string.IsNullOrWhiteSpace(endpointConfiguration.Url) + && !string.IsNullOrWhiteSpace(apiKey) + && !string.IsNullOrWhiteSpace(request.AIModel.DeploymentName) + && !string.IsNullOrWhiteSpace(apiKeyHeaderName)) { var client = _httpClientFactoryService.CreateClient(HttpClients.AzureOpenAIDirect); - if (endpointSettings.AuthenticationType == "key" && !string.IsNullOrWhiteSpace(endpointSettings.APIKey)) + if (endpointConfiguration.AuthenticationType == AuthenticationTypes.APIKey && !string.IsNullOrWhiteSpace(apiKey)) { - client.DefaultRequestHeaders.Add("api-key", endpointSettings.APIKey); + client.DefaultRequestHeaders.Add(apiKeyHeaderName, apiKey); } - client.BaseAddress = new Uri(endpointSettings.Endpoint); + client.BaseAddress = new Uri(endpointConfiguration.Url); - var modelParameters = agent.OrchestrationSettings?.ModelParameters; - var modelOverrides = request.Settings?.ModelParameters; + var modelParameters = request.AIModel.ModelParameters; - if (modelParameters != null) - { - var azureOpenAIDirectRequest = modelParameters.ToObject(modelOverrides); - var chatOperation = string.Empty; + var azureOpenAIDirectRequest = modelParameters.ToObject(); + var chatOperation = string.Empty; - switch (endpointSettings.OperationType) - { - case OperationTypes.Completions: - azureOpenAIDirectRequest.Prompt = request.UserPrompt; - break; - case OperationTypes.Chat: - chatOperation = "/chat"; - azureOpenAIDirectRequest.Messages = [.. inputStrings]; - break; - } + switch (endpointConfiguration.OperationType) + { + case OperationTypes.Completions: + azureOpenAIDirectRequest.Prompt = request.UserPrompt; + break; + case OperationTypes.Chat: + chatOperation = "/chat"; + azureOpenAIDirectRequest.Messages = [.. inputStrings]; + break; + } - if (modelOverrides != null && modelOverrides.ContainsKey(ModelParameterKeys.DeploymentName)) - { - modelParameters[ModelParameterKeys.DeploymentName] = modelOverrides[ModelParameterKeys.DeploymentName]; - } + var body = JsonSerializer.Serialize(azureOpenAIDirectRequest, _jsonSerializerOptions); + var content = new StringContent(body, Encoding.UTF8, "application/json"); - var body = JsonSerializer.Serialize(azureOpenAIDirectRequest, _jsonSerializerOptions); - var content = new StringContent(body, Encoding.UTF8, "application/json"); - modelParameters.TryGetValue(ModelParameterKeys.DeploymentName, out var deployment); + var responseMessage = await client.PostAsync($"/openai/deployments/{request.AIModel.DeploymentName}{chatOperation}/completions?api-version={endpointConfiguration.APIVersion}", content); + var responseContent = await responseMessage.Content.ReadAsStringAsync(); - var responseMessage = await client.PostAsync($"/openai/deployments/{deployment}{chatOperation}/completions?api-version={endpointSettings.APIVersion}", content); - var responseContent = await responseMessage.Content.ReadAsStringAsync(); + if (responseMessage.IsSuccessStatusCode) + { + var completionResponse = JsonSerializer.Deserialize(responseContent); - if (responseMessage.IsSuccessStatusCode) + return new LLMCompletionResponse { - var completionResponse = JsonSerializer.Deserialize(responseContent); - - return new LLMCompletionResponse - { - Completion = endpointSettings.OperationType == OperationTypes.Chat - ? completionResponse!.Choices?[0].Message?.Content - : completionResponse!.Choices?[0].Text, - UserPrompt = request.UserPrompt, - FullPrompt = body, - PromptTemplate = systemPrompt?.Content, - AgentName = agent.Name, - PromptTokens = completionResponse!.Usage!.PromptTokens, - CompletionTokens = completionResponse!.Usage!.CompletionTokens - }; - } - - _logger.LogWarning("The AzureOpenAIDirect orchestration service returned status code {StatusCode}: {ResponseContent}", - responseMessage.StatusCode, responseContent); + Completion = !string.IsNullOrEmpty(endpointConfiguration.OperationType) && endpointConfiguration.OperationType == OperationTypes.Chat + ? completionResponse!.Choices?[0].Message?.Content + : completionResponse!.Choices?[0].Text, + UserPrompt = request.UserPrompt, + FullPrompt = body, + PromptTemplate = systemPrompt?.Content, + AgentName = request.Agent.Name, + PromptTokens = completionResponse!.Usage!.PromptTokens, + CompletionTokens = completionResponse!.Usage!.CompletionTokens + }; } + + _logger.LogWarning("The AzureOpenAIDirect orchestration service returned status code {StatusCode}: {ResponseContent}", + responseMessage.StatusCode, responseContent); } return new LLMCompletionResponse @@ -176,54 +174,10 @@ public async Task GetCompletion(string instanceId, LLMCom Completion = "A problem on my side prevented me from responding.", UserPrompt = request.UserPrompt, PromptTemplate = systemPrompt?.Content, - AgentName = agent.Name, + AgentName = request.Agent.Name, PromptTokens = 0, CompletionTokens = 0 }; } - - /// - /// Extracts endpoint configuration values from a dictionary and writes them into a - /// object. - /// - /// Dictionary containing orchestration endpoint configuration values. - /// Returns a object containing the endpoint configuration. - /// - private EndpointSettings GetEndpointSettings(Dictionary endpointConfiguration) - { - if (!endpointConfiguration.TryGetValue(EndpointConfigurationKeys.Endpoint, out var endpointKeyName)) - throw new Exception("An endpoint value must be passed in via an Azure App Config key name."); - - var endpoint = _configuration.GetValue(endpointKeyName?.ToString()!); - - var authenticationType = endpointConfiguration.GetValueOrDefault(EndpointConfigurationKeys.AuthenticationType, "key").ToString(); - var apiKey = string.Empty; - - if (authenticationType == "key") - { - if (!endpointConfiguration.TryGetValue(EndpointConfigurationKeys.APIKey, out var apiKeyKeyName)) - throw new Exception("An API key must be passed in via an Azure App Config key name."); - - apiKey = _configuration.GetValue(apiKeyKeyName?.ToString()!)!; - } - - if (!endpointConfiguration.TryGetValue(EndpointConfigurationKeys.APIVersion, out var apiVersionKeyName)) - throw new Exception("An API version must be passed in via an Azure App Config key name."); - - var apiVersion = _configuration.GetValue(apiVersionKeyName?.ToString()!); - - var operationType = string.Empty; - if (endpointConfiguration.TryGetValue(EndpointConfigurationKeys.OperationType, out var operationTypeKeyName)) - operationType = _configuration.GetValue(operationTypeKeyName?.ToString()!) ?? OperationTypes.Chat; - - return new EndpointSettings - { - Endpoint = endpoint!, - APIKey = apiKey!, - APIVersion = apiVersion!, - AuthenticationType = authenticationType!, - OperationType = operationType - }; - } } } diff --git a/src/dotnet/Orchestration/Services/LLMOrchestrationServiceManager.cs b/src/dotnet/Orchestration/Services/LLMOrchestrationServiceManager.cs index 7062d00ded..ff89ce369b 100644 --- a/src/dotnet/Orchestration/Services/LLMOrchestrationServiceManager.cs +++ b/src/dotnet/Orchestration/Services/LLMOrchestrationServiceManager.cs @@ -1,4 +1,5 @@ using FoundationaLLM.Common.Authentication; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Constants.Configuration; using FoundationaLLM.Common.Constants.ResourceProviders; using FoundationaLLM.Common.Exceptions; @@ -63,17 +64,21 @@ private async Task Initialize() var configurationResourceProvider = _resourceProviderServices[ResourceProviderNames.FoundationaLLM_Configuration]; await configurationResourceProvider.WaitForInitialization(); - var apiEndpoint = await configurationResourceProvider.GetResources( + var apiEndpoint = await configurationResourceProvider.GetResources( DefaultAuthentication.ServiceIdentity!); _externalOrchestrationServiceSettings = apiEndpoint - .Where(eos => eos.APIKeyConfigurationName is not null && eos.APIKeyConfigurationName.StartsWith(AppConfigurationKeySections.FoundationaLLM_ExternalAPIs)) + .Where(ae => ae.Category == APIEndpointCategory.ExternalOrchestration + && ae.AuthenticationParameters.TryGetValue(AuthenticationParametersKeys.APIKeyConfigurationName, out var apiKeyConfigObj) + && apiKeyConfigObj is string apiKeyConfig + && !string.IsNullOrWhiteSpace(apiKeyConfig) + && apiKeyConfig.StartsWith(AppConfigurationKeySections.FoundationaLLM_APIEndpoints)) .ToDictionary( - eos => eos.Name, - eos => new APISettingsBase + ae => ae.Name, + ae => new APISettingsBase { - APIKey = _configuration[eos.APIKey], - APIUrl = _configuration[eos.Url] + APIKey = _configuration[ae.AuthenticationParameters[AuthenticationParametersKeys.APIKeyConfigurationName].ToString()!], + APIUrl = ae.Url }); _initialized = true; @@ -97,7 +102,7 @@ public async Task> GetAggregateStatus(string instanceId, .ToAsyncEnumerable() .SelectAwait(async x => await x.GetStatus(instanceId)); - await foreach(var serviceStatus in serviceStatuses) + await foreach (var serviceStatus in serviceStatuses) result.Add(serviceStatus); return result; diff --git a/src/dotnet/Orchestration/Services/OrchestrationService.cs b/src/dotnet/Orchestration/Services/OrchestrationService.cs index 0cf1edc03b..7edd05ea5b 100644 --- a/src/dotnet/Orchestration/Services/OrchestrationService.cs +++ b/src/dotnet/Orchestration/Services/OrchestrationService.cs @@ -125,6 +125,7 @@ private async Task GetCompletionForAgentConversation( var orchestration = await OrchestrationBuilder.Build( instanceId, conversationStep.AgentName, + completionRequest, _callContext, _configuration, _resourceProviderServices, @@ -141,7 +142,7 @@ private async Task GetCompletionForAgentConversation( Attachments = completionRequest.Attachments, UserPrompt = currentCompletionResponse == null ? conversationStep.UserPrompt - : $"{currentCompletionResponse.Completion}{Environment.NewLine}{conversationStep.UserPrompt}" + : $"{currentCompletionResponse.Completion}{Environment.NewLine}{conversationStep.UserPrompt}", }; currentCompletionResponse = orchestration == null diff --git a/src/dotnet/OrchestrationAPI/Program.cs b/src/dotnet/OrchestrationAPI/Program.cs index 1de23571da..80c2bedb99 100644 --- a/src/dotnet/OrchestrationAPI/Program.cs +++ b/src/dotnet/OrchestrationAPI/Program.cs @@ -50,7 +50,7 @@ public static void Main(string[] args) }); options.Select(AppConfigurationKeyFilters.FoundationaLLM_Instance); options.Select(AppConfigurationKeyFilters.FoundationaLLM_APIs); - options.Select(AppConfigurationKeyFilters.FoundationaLLM_ExternalAPIs); + options.Select(AppConfigurationKeyFilters.FoundationaLLM_APIEndpoints); options.Select(AppConfigurationKeyFilters.FoundationaLLM_Orchestration); options.Select(AppConfigurationKeyFilters.FoundationaLLM_Agent); options.Select(AppConfigurationKeyFilters.FoundationaLLM_AzureAI); @@ -61,6 +61,7 @@ public static void Main(string[] args) options.Select(AppConfigurationKeyFilters.FoundationaLLM_Configuration); options.Select(AppConfigurationKeyFilters.FoundationaLLM_DataSource); options.Select(AppConfigurationKeyFilters.FoundationaLLM_Attachment); + options.Select(AppConfigurationKeyFilters.FoundationaLLM_AIModel); }); if (builder.Environment.IsDevelopment()) builder.Configuration.AddJsonFile("appsettings.development.json", true, true); @@ -132,6 +133,7 @@ public static void Main(string[] args) builder.AddConfigurationResourceProvider(); builder.AddDataSourceResourceProvider(); builder.AddAttachmentResourceProvider(); + builder.AddAIModelResourceProvider(); // Register the downstream services and HTTP clients. RegisterDownstreamServices(builder); diff --git a/src/dotnet/SemanticKernel/Agents/SemanticKernelAgentBase.cs b/src/dotnet/SemanticKernel/Agents/SemanticKernelAgentBase.cs index 25577ac0ea..179c6d729e 100644 --- a/src/dotnet/SemanticKernel/Agents/SemanticKernelAgentBase.cs +++ b/src/dotnet/SemanticKernel/Agents/SemanticKernelAgentBase.cs @@ -1,5 +1,6 @@ -using FoundationaLLM.Common.Constants.Agents; +using FoundationaLLM.Common.Constants.Configuration; using FoundationaLLM.Common.Constants.ResourceProviders; +using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Authentication; using FoundationaLLM.Common.Models.Orchestration; @@ -8,7 +9,6 @@ using FoundationaLLM.SemanticKernel.Core.Exceptions; using Microsoft.AspNetCore.Http; using Microsoft.Extensions.Logging; -using Microsoft.SemanticKernel; namespace FoundationaLLM.SemanticKernel.Core.Agents { @@ -30,7 +30,7 @@ public class SemanticKernelAgentBase( protected readonly ILogger _logger = logger; protected string _llmProvider = string.Empty; - protected string _endpoint = string.Empty; + protected string _endpointUrl = string.Empty; protected string _deploymentName = string.Empty; /// @@ -39,13 +39,25 @@ public class SemanticKernelAgentBase( /// A object containing the completion response. public async Task GetCompletion() { - ValidateRequest(); + try + { + _request.Validate(); + _deploymentName = _request.AIModel.DeploymentName!; + _llmProvider = _request.AIModelEndpointConfiguration.Provider!; + _endpointUrl = _request.AIModelEndpointConfiguration.Url!; + + } + catch (OrchestrationException ex) + { + throw new SemanticKernelException(ex.Message, StatusCodes.Status400BadRequest); + } + await ExpandAndValidateAgent(); return _llmProvider switch { - LanguageModelProviders.MICROSOFT => await BuildResponseWithAzureOpenAI(), - LanguageModelProviders.OPENAI => await BuildResponseWithOpenAI(), + APIEndpointProviders.MICROSOFT => await BuildResponseWithAzureOpenAI(), + APIEndpointProviders.OPENAI => await BuildResponseWithOpenAI(), _ => throw new SemanticKernelException($"The LLM provider '{_llmProvider}' is not supported.") }; } @@ -101,39 +113,5 @@ private async Task GetResource(string objectId, string resourceTypeName, I var resource = (result as List>)!.First().Resource; return resource; } - - private void ValidateRequest() - { - if (_request.Agent == null) - throw new SemanticKernelException("The Agent property of the completion request cannot be null.", StatusCodes.Status400BadRequest); - - if (_request.Agent.OrchestrationSettings == null) - throw new SemanticKernelException("The OrchestrationSettings property of the agent cannot be null.", StatusCodes.Status400BadRequest); - - if (_request.Agent.OrchestrationSettings.EndpointConfiguration == null) - throw new SemanticKernelException("The EndpointConfiguration property of the agent's OrchestrationSettings property cannot be null.", StatusCodes.Status400BadRequest); - - if (!_request.Agent.OrchestrationSettings.EndpointConfiguration.TryGetValue(EndpointConfigurationKeys.Provider, out var llmProvider) - || string.IsNullOrWhiteSpace(llmProvider.ToString())) - throw new SemanticKernelException("The Provider property of the agent's OrchestrationSettings.EndpointConfiguration property cannot be null.", StatusCodes.Status400BadRequest); - - if (!LanguageModelProviders.All.Contains(llmProvider.ToString())) - throw new SemanticKernelException($"The LLM provider '{llmProvider}' is not supported.", StatusCodes.Status400BadRequest); - - if (!_request.Agent.OrchestrationSettings.EndpointConfiguration.TryGetValue(EndpointConfigurationKeys.Endpoint, out var endpoint) - || string.IsNullOrWhiteSpace(endpoint.ToString())) - throw new SemanticKernelException("The Endpoint property of the agent's OrchestrationSettings.EndpointConfiguration property cannot be null.", StatusCodes.Status400BadRequest); - - if (_request.Agent.OrchestrationSettings.ModelParameters == null) - throw new SemanticKernelException("The ModelParameters property of the agent's OrchestrationSettings property cannot be null.", StatusCodes.Status400BadRequest); - - if (!_request.Agent.OrchestrationSettings.ModelParameters.TryGetValue(ModelParameterKeys.DeploymentName, out var deploymentName) - || string.IsNullOrWhiteSpace(deploymentName.ToString())) - throw new SemanticKernelException("The DeploymentName property of the agent's OrchestrationSettings.ModelParameters property cannot be null.", StatusCodes.Status400BadRequest); - - _llmProvider = llmProvider.ToString()!; - _endpoint = endpoint.ToString()!; - _deploymentName = deploymentName.ToString()!; - } } } diff --git a/src/dotnet/SemanticKernel/Agents/SemanticKernelKnowledgeManagementAgent.cs b/src/dotnet/SemanticKernel/Agents/SemanticKernelKnowledgeManagementAgent.cs index b4ea4b8cb9..0f5b4fb7a0 100644 --- a/src/dotnet/SemanticKernel/Agents/SemanticKernelKnowledgeManagementAgent.cs +++ b/src/dotnet/SemanticKernel/Agents/SemanticKernelKnowledgeManagementAgent.cs @@ -25,6 +25,7 @@ using Microsoft.SemanticKernel.Connectors.AzureCosmosDBNoSQL; using Microsoft.SemanticKernel.Connectors.Postgres; using System.Runtime; +using FoundationaLLM.Common.Constants.Authentication; #pragma warning disable SKEXP0001, SKEXP0010, SKEXP0020, SKEXP0050, SKEXP0060 @@ -57,91 +58,34 @@ public class SemanticKernelKnowledgeManagementAgent( protected override async Task ExpandAndValidateAgent() { - var agent = _request.Agent as KnowledgeManagementAgent; - - if (agent!.OrchestrationSettings!.AgentParameters == null) - throw new SemanticKernelException("The agent parameters are missing in the orchestration settings.", StatusCodes.Status400BadRequest); - - #region Other agent descriptions - - if (agent.OrchestrationSettings.AgentParameters.TryGetValue( - "AllAgents", out var allAgentDescriptions)) - { - _agentDescriptions = allAgentDescriptions is JsonElement allAgentDescriptionsJsonElement - ? allAgentDescriptionsJsonElement.Deserialize>() - : allAgentDescriptions as Dictionary; - } - - #endregion - - #region Prompt - - if (string.IsNullOrWhiteSpace(agent.PromptObjectId)) - throw new SemanticKernelException("Invalid prompt object id.", StatusCodes.Status400BadRequest); - - if (!agent.OrchestrationSettings.AgentParameters.TryGetValue( - agent.PromptObjectId, out var promptObject)) - throw new SemanticKernelException("The prompt object is missing from the agent parameters.", StatusCodes.Status400BadRequest); - - var prompt = promptObject is JsonElement promptJsonElement - ? promptJsonElement.Deserialize() - : promptObject as MultipartPrompt; - - if (prompt == null - || string.IsNullOrWhiteSpace(prompt.Prefix)) - throw new SemanticKernelException("The prompt object provided in the agent parameters is invalid.", StatusCodes.Status400BadRequest); - - _prompt = prompt.Prefix; - - #endregion + _agentDescriptions = _request.OtherAgentsDescriptions; + _prompt = _request.Prompt.Prefix!; #region Vectorization (text embedding and indexing) - optional - if (!string.IsNullOrWhiteSpace(agent.Vectorization.TextEmbeddingProfileObjectId)) - { - if (!agent.OrchestrationSettings.AgentParameters.TryGetValue( - agent.Vectorization.TextEmbeddingProfileObjectId, out var textEmbeddingProfileObject)) - throw new SemanticKernelException("The text embedding profile object is missing from the agent parameters.", StatusCodes.Status400BadRequest); - - var textEmbeddingProfile = textEmbeddingProfileObject is JsonElement textEmbeddingProfileJsonElement - ? textEmbeddingProfileJsonElement.Deserialize() - : textEmbeddingProfileObject as TextEmbeddingProfile; - - if (textEmbeddingProfile == null - || textEmbeddingProfile.ConfigurationReferences == null - || !textEmbeddingProfile.ConfigurationReferences.TryGetValue("DeploymentName", out var deploymentNameConfigurationItem) - || string.IsNullOrWhiteSpace(deploymentNameConfigurationItem) - || !textEmbeddingProfile.ConfigurationReferences.TryGetValue("Endpoint", out var textEmbeddingEndpointConfigurationItem) - || string.IsNullOrWhiteSpace(textEmbeddingEndpointConfigurationItem)) - throw new SemanticKernelException("The text embedding profile object provided in the agent parameters is invalid.", StatusCodes.Status400BadRequest); + var textEmbeddingProfile = _request.TextEmbeddingProfile; + var indexingProfiles = _request.IndexingProfiles; + if (textEmbeddingProfile != null) + { _textEmbeddingDeploymentName = textEmbeddingProfile.Settings != null && textEmbeddingProfile.Settings.TryGetValue("deployment_name", out string? deploymentNameOverride) && !string.IsNullOrWhiteSpace(deploymentNameOverride) ? deploymentNameOverride - : await GetConfigurationValue(deploymentNameConfigurationItem); - _textEmbeddingEndpoint = await GetConfigurationValue(textEmbeddingEndpointConfigurationItem); + : await GetConfigurationValue(textEmbeddingProfile.ConfigurationReferences!["DeploymentName"]); + _textEmbeddingEndpoint = await GetConfigurationValue(textEmbeddingProfile.ConfigurationReferences!["EndpointUrl"]); } - if ((agent.Vectorization.IndexingProfileObjectIds ?? []).Count > 0) + if ((indexingProfiles ?? []).Count > 0) { - if (string.IsNullOrEmpty(agent.Vectorization.IndexingProfileObjectIds[0])) - throw new SemanticKernelException("The indexing profile object is missing from the agent parameters.", StatusCodes.Status400BadRequest); - - if (!agent.OrchestrationSettings.AgentParameters.TryGetValue( - agent.Vectorization.IndexingProfileObjectIds[0], out var indexingProfileObject)) - throw new SemanticKernelException("The indexing profile object is missing from the agent parameters.", StatusCodes.Status400BadRequest); - - var indexingProfile = indexingProfileObject is JsonElement indexingProfileJsonElement - ? indexingProfileJsonElement.Deserialize() - : indexingProfileObject as IndexingProfile; + var indexingProfile = indexingProfiles![0]; if (indexingProfile == null || !await ValidateAndMapIndexingProfileConfiguration(indexingProfile) || indexingProfile.Settings == null || !indexingProfile.Settings.TryGetValue("IndexName", out var indexName) || string.IsNullOrWhiteSpace(indexName)) - throw new SemanticKernelException("The indexing profile object provided in the agent parameters is invalid.", StatusCodes.Status400BadRequest); + throw new SemanticKernelException("The indexing profile object provided in the request's objects is invalid.", StatusCodes.Status400BadRequest); _indexerName = indexingProfile.Indexer.ToString(); _indexName = indexName; @@ -168,7 +112,7 @@ protected async Task ValidateAndMapIndexingProfileConfiguration(IndexingPr { case IndexerType.AzureAISearchIndexer: valid = indexingProfile.ConfigurationReferences != null - && indexingProfile.ConfigurationReferences.TryGetValue("Endpoint", + && indexingProfile.ConfigurationReferences.TryGetValue("EndpointUrl", out indexingEndpointConfigurationItem) && !string.IsNullOrWhiteSpace(indexingEndpointConfigurationItem) && indexingProfile.ConfigurationReferences.TryGetValue("AuthenticationType", @@ -179,7 +123,7 @@ protected async Task ValidateAndMapIndexingProfileConfiguration(IndexingPr _azureAISearchIndexingServiceSettings = new AzureAISearchIndexingServiceSettings { Endpoint = await GetConfigurationValue(indexingEndpointConfigurationItem!), - AuthenticationType = Enum.Parse(await GetConfigurationValue(authenticationType!)) + AuthenticationType = Enum.Parse(await GetConfigurationValue(authenticationType!)) }; } break; @@ -251,7 +195,6 @@ protected override async Task BuildResponseWithAzureOpenA AgentName = _request.Agent.Name, PromptTokens = completionUsage!.PromptTokens, CompletionTokens = completionUsage.CompletionTokens, - TotalTokens = completionUsage.TotalTokens }; } catch (Exception ex) @@ -274,7 +217,7 @@ private Kernel BuildKernel() builder.AddAzureOpenAIChatCompletion( _deploymentName, - _endpoint, + _endpointUrl, credential, null, null, diff --git a/src/dotnet/SemanticKernel/Models/Configuration/AzureAISearchIndexingServiceSettings.cs b/src/dotnet/SemanticKernel/Models/Configuration/AzureAISearchIndexingServiceSettings.cs index 96cdb59053..9a36b338ba 100644 --- a/src/dotnet/SemanticKernel/Models/Configuration/AzureAISearchIndexingServiceSettings.cs +++ b/src/dotnet/SemanticKernel/Models/Configuration/AzureAISearchIndexingServiceSettings.cs @@ -1,4 +1,5 @@ -using FoundationaLLM.Common.Settings; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Settings; using System; using System.Collections.Generic; using System.Linq; @@ -22,6 +23,6 @@ public record AzureAISearchIndexingServiceSettings /// The indicating which authentication mechanism to use. /// [JsonConverter(typeof(JsonStringEnumConverter))] - public required AzureAISearchAuthenticationTypes AuthenticationType { get; set; } + public required AuthenticationTypes AuthenticationType { get; set; } } } diff --git a/src/dotnet/SemanticKernel/Models/Configuration/SemanticKernelTextEmbeddingServiceSettings.cs b/src/dotnet/SemanticKernel/Models/Configuration/SemanticKernelTextEmbeddingServiceSettings.cs index 538a54da34..ba52fd5df1 100644 --- a/src/dotnet/SemanticKernel/Models/Configuration/SemanticKernelTextEmbeddingServiceSettings.cs +++ b/src/dotnet/SemanticKernel/Models/Configuration/SemanticKernelTextEmbeddingServiceSettings.cs @@ -1,4 +1,5 @@ -using FoundationaLLM.Common.Settings; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Settings; using System.Text.Json.Serialization; namespace FoundationaLLM.SemanticKernel.Core.Models.Configuration @@ -27,6 +28,6 @@ public record SemanticKernelTextEmbeddingServiceSettings /// The indicating which authentication mechanism to use. /// [JsonConverter(typeof(JsonStringEnumConverter))] - public required AzureOpenAIAuthenticationTypes AuthenticationType { get; set; } + public required AuthenticationTypes AuthenticationType { get; set; } } } diff --git a/src/dotnet/SemanticKernel/Services/Indexing/AzureAISearchIndexingService.cs b/src/dotnet/SemanticKernel/Services/Indexing/AzureAISearchIndexingService.cs index 434353f9a1..06356869f2 100644 --- a/src/dotnet/SemanticKernel/Services/Indexing/AzureAISearchIndexingService.cs +++ b/src/dotnet/SemanticKernel/Services/Indexing/AzureAISearchIndexingService.cs @@ -1,4 +1,5 @@ using FoundationaLLM.Common.Authentication; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Vectorization; @@ -91,7 +92,7 @@ private AzureAISearchMemoryStore CreateMemoryStore() { switch (_settings.AuthenticationType) { - case AzureAISearchAuthenticationTypes.AzureIdentity: + case AuthenticationTypes.AzureIdentity: ValidateEndpoint(_settings.Endpoint); return CreateMemoryStoreFromIdentity(_settings.Endpoint); default: diff --git a/src/dotnet/SemanticKernel/Services/SemanticKernelTextEmbeddingService.cs b/src/dotnet/SemanticKernel/Services/SemanticKernelTextEmbeddingService.cs index 8adebc8357..125187abf0 100644 --- a/src/dotnet/SemanticKernel/Services/SemanticKernelTextEmbeddingService.cs +++ b/src/dotnet/SemanticKernel/Services/SemanticKernelTextEmbeddingService.cs @@ -1,4 +1,5 @@ using FoundationaLLM.Common.Authentication; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Exceptions; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Vectorization; @@ -81,7 +82,7 @@ private Kernel CreateKernel() ValidateEndpoint(_settings.Endpoint); var builder = Kernel.CreateBuilder(); - if (_settings.AuthenticationType == AzureOpenAIAuthenticationTypes.AzureIdentity) + if (_settings.AuthenticationType == AuthenticationTypes.AzureIdentity) { builder.AddAzureOpenAITextEmbeddingGeneration( _settings.DeploymentName, diff --git a/src/dotnet/Vectorization/Services/ContentSources/ContentSourceServiceFactory.cs b/src/dotnet/Vectorization/Services/ContentSources/ContentSourceServiceFactory.cs index 8dcf7b992b..0425de3a64 100644 --- a/src/dotnet/Vectorization/Services/ContentSources/ContentSourceServiceFactory.cs +++ b/src/dotnet/Vectorization/Services/ContentSources/ContentSourceServiceFactory.cs @@ -1,3 +1,4 @@ +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Constants.Configuration; using FoundationaLLM.Common.Constants.ResourceProviders; using FoundationaLLM.Common.Exceptions; @@ -62,7 +63,7 @@ public IContentSourceService GetService(string serviceName) private DataLakeContentSourceService CreateAzureDataLakeContentSourceService(string serviceName) { - var blobStorageServiceSettings = new BlobStorageServiceSettings { AuthenticationType = BlobStorageAuthenticationTypes.Unknown }; + var blobStorageServiceSettings = new BlobStorageServiceSettings { AuthenticationType = AuthenticationTypes.Unknown }; _configuration.Bind( $"{AppConfigurationKeySections.FoundationaLLM_DataSources}:{serviceName}", blobStorageServiceSettings); diff --git a/src/dotnet/Vectorization/Services/Pipelines/PipelineExecutionService.cs b/src/dotnet/Vectorization/Services/Pipelines/PipelineExecutionService.cs index 4d449c4595..a073129609 100644 --- a/src/dotnet/Vectorization/Services/Pipelines/PipelineExecutionService.cs +++ b/src/dotnet/Vectorization/Services/Pipelines/PipelineExecutionService.cs @@ -19,6 +19,7 @@ using Quartz; using System.Configuration; using FoundationaLLM.Vectorization.Services.DataSources.Configuration.SQLDatabase; +using FoundationaLLM.Common.Constants.Authentication; namespace FoundationaLLM.Vectorization.Services.Pipelines { @@ -122,7 +123,7 @@ public async Task ExecuteAsync(CancellationToken cancellationToken) { case DataSourceTypes.AzureDataLake: // resolve configuration references - var blobStorageServiceSettings = new BlobStorageServiceSettings { AuthenticationType = BlobStorageAuthenticationTypes.Unknown }; + var blobStorageServiceSettings = new BlobStorageServiceSettings { AuthenticationType = AuthenticationTypes.Unknown }; _configuration.Bind( $"{AppConfigurationKeySections.FoundationaLLM_DataSources}:{dataSource.Name}", blobStorageServiceSettings); diff --git a/src/dotnet/Vectorization/Services/Text/TextEmbeddingServiceFactory.cs b/src/dotnet/Vectorization/Services/Text/TextEmbeddingServiceFactory.cs index 194c818777..935fe9e831 100644 --- a/src/dotnet/Vectorization/Services/Text/TextEmbeddingServiceFactory.cs +++ b/src/dotnet/Vectorization/Services/Text/TextEmbeddingServiceFactory.cs @@ -1,3 +1,4 @@ +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Constants.Configuration; using FoundationaLLM.Common.Constants.ResourceProviders; using FoundationaLLM.Common.Exceptions; @@ -74,9 +75,9 @@ public ITextEmbeddingService GetService(string serviceName) private ITextEmbeddingService CreateSemanticKernelTextEmbeddingService(TextEmbeddingProfile textEmbeddingProfile) { - if (!textEmbeddingProfile.ConfigurationReferences!.TryGetValue("Endpoint", out string? endpointConfigurationItem) + if (!textEmbeddingProfile.ConfigurationReferences!.TryGetValue("EndpointUrl", out string? endpointConfigurationItem) || string.IsNullOrWhiteSpace(endpointConfigurationItem)) - throw new VectorizationException("The text embedding profile does not contain a valid Endpoint configuration reference."); + throw new VectorizationException("The text embedding profile does not contain a valid EndpointUrl configuration reference."); if (!textEmbeddingProfile.ConfigurationReferences!.TryGetValue("DeploymentName", out string? deploymentNameConfigurationItem) || string.IsNullOrWhiteSpace(deploymentNameConfigurationItem)) @@ -91,7 +92,7 @@ private ITextEmbeddingService CreateSemanticKernelTextEmbeddingService(TextEmbed return new SemanticKernelTextEmbeddingService( Options.Create(new SemanticKernelTextEmbeddingServiceSettings { - AuthenticationType = AzureOpenAIAuthenticationTypes.AzureIdentity, + AuthenticationType = AuthenticationTypes.AzureIdentity, Endpoint = _configuration[endpointConfigurationItem]!, DeploymentName = deploymentName! }), diff --git a/tests/dotnet/Common.Tests/Models/Agents/AgentBaseTests.cs b/tests/dotnet/Common.Tests/Models/Agents/AgentBaseTests.cs index 9460adb703..e91e5fbbe9 100644 --- a/tests/dotnet/Common.Tests/Models/Agents/AgentBaseTests.cs +++ b/tests/dotnet/Common.Tests/Models/Agents/AgentBaseTests.cs @@ -38,22 +38,22 @@ public void AgentType_UnsupportedType_ThrowsException() public void ConversationHistory_SetAndGet_ReturnsCorrectValue() { // Arrange - var conversationHistory = new ConversationHistory { Enabled = true, MaxHistory = 100 }; - _agentBase.ConversationHistory = conversationHistory; + var conversationHistory = new ConversationHistorySettings { Enabled = true, MaxHistory = 100 }; + _agentBase.ConversationHistorySettings = conversationHistory; // Assert - Assert.Equal(conversationHistory, _agentBase.ConversationHistory); + Assert.Equal(conversationHistory, _agentBase.ConversationHistorySettings); } [Fact] public void Gatekeeper_SetAndGet_ReturnsCorrectValue() { // Arrange - var gatekeeper = new Gatekeeper { UseSystemSetting = false, Options = new string[] { "Option1", "Option2" } }; - _agentBase.Gatekeeper = gatekeeper; + var gatekeeper = new GatekeeperSettings { UseSystemSetting = false, Options = new string[] { "Option1", "Option2" } }; + _agentBase.GatekeeperSettings = gatekeeper; // Assert - Assert.Equal(gatekeeper, _agentBase.Gatekeeper); + Assert.Equal(gatekeeper, _agentBase.GatekeeperSettings); } } diff --git a/tests/dotnet/Common.Tests/Models/Orchestration/InternalContextCompletionRequestTests.cs b/tests/dotnet/Common.Tests/Models/Orchestration/InternalContextCompletionRequestTests.cs index c58f4de2a5..03f5216c38 100644 --- a/tests/dotnet/Common.Tests/Models/Orchestration/InternalContextCompletionRequestTests.cs +++ b/tests/dotnet/Common.Tests/Models/Orchestration/InternalContextCompletionRequestTests.cs @@ -10,7 +10,7 @@ public void InternalContextCompletionRequest_Agent_Property_Test() { // Arrange var request = new LLMCompletionRequest() - { Agent = new InternalContextAgent() { Name = "Test_agent", ObjectId = "Test_objectid", Type = AgentTypes.InternalContext} }; + { Agent = new InternalContextAgent() { Name = "Test_agent", ObjectId = "Test_objectid", Type = AgentTypes.InternalContext }, UserPrompt = "" }; var agent = new InternalContextAgent() { Name = "Test_agent", ObjectId = "Test_objectid", Type = AgentTypes.InternalContext }; diff --git a/tests/dotnet/Common.Tests/Models/Orchestration/KnowledgeManagementCompletionRequestTests.cs b/tests/dotnet/Common.Tests/Models/Orchestration/KnowledgeManagementCompletionRequestTests.cs index 8b10cc2ce0..2f712be66d 100644 --- a/tests/dotnet/Common.Tests/Models/Orchestration/KnowledgeManagementCompletionRequestTests.cs +++ b/tests/dotnet/Common.Tests/Models/Orchestration/KnowledgeManagementCompletionRequestTests.cs @@ -10,7 +10,7 @@ public void KnowledgeManagementCompletionRequest_Agent_Property_Test() { // Arrange var request = new LLMCompletionRequest() - { Agent = new KnowledgeManagementAgent() { Name = "Test_agent", ObjectId = "Test_objectid", Type = AgentTypes.KnowledgeManagement } }; + { UserPrompt="", Agent = new KnowledgeManagementAgent() { Name = "Test_agent", ObjectId = "Test_objectid", Type = AgentTypes.KnowledgeManagement } }; var agent = new KnowledgeManagementAgent() { Name = "Test_agent", ObjectId = "Test_objectid", Type = AgentTypes.KnowledgeManagement }; diff --git a/tests/dotnet/Common.Tests/Models/Orchestration/LLMCompletionResponseTests.cs b/tests/dotnet/Common.Tests/Models/Orchestration/LLMCompletionResponseTests.cs index 4feb26b5ac..3ad530b9d9 100644 --- a/tests/dotnet/Common.Tests/Models/Orchestration/LLMCompletionResponseTests.cs +++ b/tests/dotnet/Common.Tests/Models/Orchestration/LLMCompletionResponseTests.cs @@ -17,7 +17,6 @@ public void LLMCompletionResponse_Properties_Test() AgentName = "Agent name", PromptTokens = 10, CompletionTokens = 20, - TotalTokens = 30, TotalCost = 15.5f }; diff --git a/tests/dotnet/Core.Examples/Catalogs/AIModelCatalog.cs b/tests/dotnet/Core.Examples/Catalogs/AIModelCatalog.cs new file mode 100644 index 0000000000..c7159d8409 --- /dev/null +++ b/tests/dotnet/Core.Examples/Catalogs/AIModelCatalog.cs @@ -0,0 +1,44 @@ +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Core.Examples.Constants; + +namespace FoundationaLLM.Core.Examples.Catalogs +{ + public static class AIModelCatalog + { + public static readonly List CompletionAIModels = + [ + new CompletionAIModel { + Name = TestAIModelNames.Completions_Default, + Description = "The default completions AI model.", + EndpointObjectId = TestAPIEndpointConfigurationNames.DefaultAzureOpenAI, // must be filled in during the test environment setup + DeploymentName = TestAIModelNames.Completions_Default, + ModelParameters = new Dictionary + { + { "temperature", 0 }, + } + }, + new CompletionAIModel { + Name = TestAIModelNames.Completions_GPT4_32K, + Description = "The GPT-4 32K completions AI model.", + EndpointObjectId = TestAPIEndpointConfigurationNames.DefaultAzureOpenAI, // must be filled in during the test environment setup + DeploymentName = TestAIModelNames.Completions_GPT4_32K, + ModelParameters = new Dictionary + { + { "temperature", 0 }, + } + } + ]; + + /// + /// Retrieves all API endpoint configurations defined in the catalog. + /// + /// + public static List GetAllAIModels() + { + var aiModels = new List(); + aiModels.AddRange(CompletionAIModels); + + return aiModels; + } + } +} diff --git a/tests/dotnet/Core.Examples/Catalogs/APIEndpointConfigurationCatalog.cs b/tests/dotnet/Core.Examples/Catalogs/APIEndpointConfigurationCatalog.cs new file mode 100644 index 0000000000..49b2eb22c9 --- /dev/null +++ b/tests/dotnet/Core.Examples/Catalogs/APIEndpointConfigurationCatalog.cs @@ -0,0 +1,36 @@ +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Models.ResourceProviders.Configuration; +using FoundationaLLM.Common.Models.ResourceProviders.Vectorization; +using FoundationaLLM.Core.Examples.Constants; + +namespace FoundationaLLM.Core.Examples.Catalogs +{ + public static class APIEndpointConfigurationCatalog + { + public static readonly List APIEndpointConfigurations = + [ + new APIEndpointConfiguration + { + Name = TestAPIEndpointConfigurationNames.DefaultAzureOpenAI, + Description = "The default Azure OpenAI endpoint.", + Category = APIEndpointCategory.LLM, + AuthenticationType = AuthenticationTypes.APIKey, + Url = "FoundationaLLM:AzureOpenAI:API:Endpoint", // must be filled in during the test environment setup + TimeoutSeconds = 1800, + RetryStrategyName = TestRetryStrategyNames.ExponentialBackoff + } + ]; + + /// + /// Retrieves all API endpoint configurations defined in the catalog. + /// + /// + public static List GetAllAPIEndpointConfigurations() + { + var apiEndpointConfigurations = new List(); + apiEndpointConfigurations.AddRange(APIEndpointConfigurations); + + return apiEndpointConfigurations; + } + } +} diff --git a/tests/dotnet/Core.Examples/Catalogs/AgentCatalog.cs b/tests/dotnet/Core.Examples/Catalogs/AgentCatalog.cs index bf1dfc8960..1e55bc5f6b 100644 --- a/tests/dotnet/Core.Examples/Catalogs/AgentCatalog.cs +++ b/tests/dotnet/Core.Examples/Catalogs/AgentCatalog.cs @@ -1,6 +1,10 @@ using FoundationaLLM.Common.Constants; -using FoundationaLLM.Common.Models.Orchestration; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Constants.Configuration; using FoundationaLLM.Common.Models.ResourceProviders.Agent; +using FoundationaLLM.Common.Models.ResourceProviders.AIModel; +using FoundationaLLM.Common.Models.ResourceProviders.Configuration; +using FoundationaLLM.Common.Models.ResourceProviders.Vectorization; using FoundationaLLM.Core.Examples.Constants; namespace FoundationaLLM.Core.Examples.Catalogs @@ -12,6 +16,7 @@ namespace FoundationaLLM.Core.Examples.Catalogs public static class AgentCatalog { #region Knowledge Management agents + /// /// Catalog of knowledge management agents. /// @@ -29,32 +34,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings - { - Orchestrator = LLMOrchestrationServiceNames.LangChain, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + OrchestrationSettings = new AgentOrchestrationSettings + { + Orchestrator = LLMOrchestrationServiceNames.LangChain + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -68,32 +61,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null, }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings + OrchestrationSettings = new AgentOrchestrationSettings { Orchestrator = LLMOrchestrationServiceNames.SemanticKernel, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -108,32 +89,20 @@ public static class AgentCatalog TextEmbeddingProfileObjectId = null, DataSourceObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings + OrchestrationSettings = new AgentOrchestrationSettings { - Orchestrator = LLMOrchestrationServiceNames.SemanticKernel, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + Orchestrator = LLMOrchestrationServiceNames.SemanticKernel + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -148,32 +117,20 @@ public static class AgentCatalog TextEmbeddingProfileObjectId = null, DataSourceObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings - { - Orchestrator = LLMOrchestrationServiceNames.LangChain, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + OrchestrationSettings = new AgentOrchestrationSettings + { + Orchestrator = LLMOrchestrationServiceNames.LangChain + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -187,32 +144,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings + OrchestrationSettings = new AgentOrchestrationSettings { - Orchestrator = LLMOrchestrationServiceNames.SemanticKernel, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + Orchestrator = LLMOrchestrationServiceNames.SemanticKernel + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -226,32 +171,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings - { - Orchestrator = LLMOrchestrationServiceNames.LangChain, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + OrchestrationSettings = new AgentOrchestrationSettings + { + Orchestrator = LLMOrchestrationServiceNames.LangChain + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -265,32 +198,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings - { - Orchestrator = LLMOrchestrationServiceNames.LangChain, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0.5 }, - { "deployment_name", "completions-gpt-4-32k" } - } - } + OrchestrationSettings = new AgentOrchestrationSettings + { + Orchestrator = LLMOrchestrationServiceNames.LangChain + }, + AIModelObjectId = TestAIModelNames.Completions_GPT4_32K }, new KnowledgeManagementAgent { @@ -304,32 +225,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings + OrchestrationSettings = new AgentOrchestrationSettings { - Orchestrator = LLMOrchestrationServiceNames.SemanticKernel, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + Orchestrator = LLMOrchestrationServiceNames.SemanticKernel + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -343,32 +252,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings + OrchestrationSettings = new AgentOrchestrationSettings { - Orchestrator = LLMOrchestrationServiceNames.SemanticKernel, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + Orchestrator = LLMOrchestrationServiceNames.SemanticKernel + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -382,32 +279,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings + OrchestrationSettings = new AgentOrchestrationSettings { - Orchestrator = LLMOrchestrationServiceNames.SemanticKernel, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + Orchestrator = LLMOrchestrationServiceNames.SemanticKernel + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, new KnowledgeManagementAgent { @@ -421,32 +306,20 @@ public static class AgentCatalog IndexingProfileObjectIds = null, TextEmbeddingProfileObjectId = null }, - ConversationHistory = new ConversationHistory + ConversationHistorySettings = new ConversationHistorySettings { Enabled = true, MaxHistory = 10 }, - Gatekeeper = new Gatekeeper + GatekeeperSettings = new GatekeeperSettings { UseSystemSetting = false }, - OrchestrationSettings = new OrchestrationSettings - { - Orchestrator = LLMOrchestrationServiceNames.LangChain, - EndpointConfiguration = new Dictionary - { - { "auth_type", "key" }, - { "provider", "microsoft" }, - { "endpoint", "FoundationaLLM:AzureOpenAI:API:Endpoint" }, - { "api_key", "FoundationaLLM:AzureOpenAI:API:Key" }, - { "api_version", "FoundationaLLM:AzureOpenAI:API:Version" } - }, - ModelParameters = new Dictionary - { - { "temperature", 0 }, - { "deployment_name", "completions" } - } - } + OrchestrationSettings = new AgentOrchestrationSettings + { + Orchestrator = LLMOrchestrationServiceNames.LangChain + }, + AIModelObjectId = TestAIModelNames.Completions_Default }, ]; @@ -460,7 +333,7 @@ public static List GetAllAgents() { var agents = new List(); agents.AddRange(KnowledgeManagementAgents); - + return agents; } } diff --git a/tests/dotnet/Core.Examples/Constants/TestAIModelNames.cs b/tests/dotnet/Core.Examples/Constants/TestAIModelNames.cs new file mode 100644 index 0000000000..8372af49d7 --- /dev/null +++ b/tests/dotnet/Core.Examples/Constants/TestAIModelNames.cs @@ -0,0 +1,18 @@ +namespace FoundationaLLM.Core.Examples.Constants +{ + /// + /// Contains constants for test AI model names. + /// + public static class TestAIModelNames + { + /// + /// The name of the default completions AI model. + /// + public const string Completions_Default = "completions"; + + /// + /// The name of the GPT-4-32K completions AI model. + /// + public const string Completions_GPT4_32K = "completions-gpt-4-32k"; + } +} diff --git a/tests/dotnet/Core.Examples/Constants/TestAPIEndpointConfigurationNames.cs b/tests/dotnet/Core.Examples/Constants/TestAPIEndpointConfigurationNames.cs new file mode 100644 index 0000000000..24b0cc8f9a --- /dev/null +++ b/tests/dotnet/Core.Examples/Constants/TestAPIEndpointConfigurationNames.cs @@ -0,0 +1,13 @@ +namespace FoundationaLLM.Core.Examples.Constants +{ + /// + /// Contains constants for test API endpoint configuration names. + /// + public static class TestAPIEndpointConfigurationNames + { + /// + /// The name of the default Azure OpenAI API endpoint configuration. + /// + public const string DefaultAzureOpenAI = "DefaultAzureOpenAI"; + } +} diff --git a/tests/dotnet/Core.Examples/Constants/TestRetryStrategyNames.cs b/tests/dotnet/Core.Examples/Constants/TestRetryStrategyNames.cs new file mode 100644 index 0000000000..55f9cde26a --- /dev/null +++ b/tests/dotnet/Core.Examples/Constants/TestRetryStrategyNames.cs @@ -0,0 +1,13 @@ +namespace FoundationaLLM.Core.Examples.Constants +{ + /// + /// Contains constants for test retry strategy names. + /// + public static class TestRetryStrategyNames + { + /// + /// The name of exponential backoff retry strategy. + /// + public const string ExponentialBackoff = "ExponentialBackoff"; + } +} diff --git a/tests/dotnet/Core.Examples/Services/ManagementAPITestManager.cs b/tests/dotnet/Core.Examples/Services/ManagementAPITestManager.cs index db2b78baf8..9682145ee8 100644 --- a/tests/dotnet/Core.Examples/Services/ManagementAPITestManager.cs +++ b/tests/dotnet/Core.Examples/Services/ManagementAPITestManager.cs @@ -222,18 +222,19 @@ public async Task CreateAgent(string agentName, string? indexingProfi throw new InvalidOperationException($"The agent {agentName} was not found."); } + // TODO: we need support for creating APIEndpointConfiguration and AIModel object in ManagementClient + // This will break everything in the E2E tests. + // Resolve App Config values for the endpoint configuration as necessary. // Note: This is a temporary workaround until we have the Models and Endpoints resource provider in place. - if (agent.OrchestrationSettings is {EndpointConfiguration: not null}) - { - foreach (var (key, value) in agent.OrchestrationSettings.EndpointConfiguration) - { - if (key.ToLower() == "api_key") continue; - if (value is not string stringValue || !stringValue.StartsWith("FoundationaLLM:")) continue; - var appConfigValue = await TestConfiguration.GetAppConfigValueAsync(value.ToString()!); - agent.OrchestrationSettings.EndpointConfiguration[key] = appConfigValue; - } - } + //var endpoint = agent.OrchestrationSettings?.AIModel?.Endpoint; + //if (endpoint != null) + //{ + // if (endpoint.Url != null && endpoint.Url.StartsWith("FoundationaLLM:")) + // endpoint.Url = await TestConfiguration.GetAppConfigValueAsync(endpoint.Url!); + // if (endpoint.APIVersion != null && endpoint.APIVersion.StartsWith("FoundationaLLM:")) + // endpoint.APIVersion = await TestConfiguration.GetAppConfigValueAsync(endpoint.APIVersion!); + //} var agentPrompt = await CreatePrompt(agentName); // Add the prompt ObjectId to the agent. diff --git a/tests/dotnet/Core.Examples/Services/VectorizationTestService.cs b/tests/dotnet/Core.Examples/Services/VectorizationTestService.cs index d8176fc8e6..280cc80d61 100644 --- a/tests/dotnet/Core.Examples/Services/VectorizationTestService.cs +++ b/tests/dotnet/Core.Examples/Services/VectorizationTestService.cs @@ -79,7 +79,7 @@ async public Task QueryIndex(string indexProfileName, string e public async Task> GetVector(TextEmbeddingProfile embedProfile, string query) { //embed the query - string oaiEndpoint = await TestConfiguration.GetAppConfigValueAsync(embedProfile.ConfigurationReferences["Endpoint"]); + string oaiEndpoint = await TestConfiguration.GetAppConfigValueAsync(embedProfile.ConfigurationReferences["EndpointUrl"]); string authType = await TestConfiguration.GetAppConfigValueAsync(embedProfile.ConfigurationReferences["AuthenticationType"]); OpenAIClient openAIClient; switch(authType) @@ -107,7 +107,7 @@ public async Task> GetVector(TextEmbeddingProfile embedPro async public Task GetIndexClient(IndexingProfile indexProfile) { - string searchServiceEndPoint = await TestConfiguration.GetAppConfigValueAsync(indexProfile.ConfigurationReferences["Endpoint"]); + string searchServiceEndPoint = await TestConfiguration.GetAppConfigValueAsync(indexProfile.ConfigurationReferences["EndpointUrl"]); string authType = await TestConfiguration.GetAppConfigValueAsync(indexProfile.ConfigurationReferences["AuthenticationType"]); SearchIndexClient indexClient = null; diff --git a/tests/dotnet/Management.Client.Tests/Clients/Resources/ConfigurationManagementClientTests.cs b/tests/dotnet/Management.Client.Tests/Clients/Resources/ConfigurationManagementClientTests.cs index dcf44fd77f..06f36d97ec 100644 --- a/tests/dotnet/Management.Client.Tests/Clients/Resources/ConfigurationManagementClientTests.cs +++ b/tests/dotnet/Management.Client.Tests/Clients/Resources/ConfigurationManagementClientTests.cs @@ -1,5 +1,6 @@ using FoundationaLLM.Client.Management.Clients.Resources; using FoundationaLLM.Client.Management.Interfaces; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Constants.ResourceProviders; using FoundationaLLM.Common.Models.ResourceProviders; using FoundationaLLM.Common.Models.ResourceProviders.Configuration; @@ -101,20 +102,23 @@ await _mockRestClient.Resources.Received(1).GetResourcesAsync> + var expectedServices = new List> { - new ResourceProviderGetResult + new ResourceProviderGetResult { - Resource = new APIEndpoint + Resource = new APIEndpointConfiguration { Name = "test-service", Url = "FoundationaLLM:TestAPIUrlConfiguration", - APIKeyConfigurationName = "FoundationaLLM:TestAPIKeyConfiguration", Category = APIEndpointCategory.General, - AuthenticationType = "APIKey", - APIKeyHeaderName = "FoundationaLLM:TestAPIKeyHeaderName", + AuthenticationType = AuthenticationTypes.APIKey, TimeoutSeconds = 60, - RetryStrategyName = "ExponentialBackoff" + RetryStrategyName = "ExponentialBackoff", + AuthenticationParameters = + { + { AuthenticationParametersKeys.APIKeyConfigurationName, "FoundationaLLM:TestAPIKeyConfiguration" }, + {AuthenticationParametersKeys.APIKeyHeaderName, "FoundationaLLM:TestAPIKeyHeaderName" } + } }, Actions = [], Roles = [] @@ -122,9 +126,9 @@ public async Task GetAPIEndpointsAsync_ShouldReturnServices() }; _mockRestClient.Resources - .GetResourcesAsync>>( + .GetResourcesAsync>>( ResourceProviderNames.FoundationaLLM_Configuration, - ConfigurationResourceTypeNames.APIEndpoints + ConfigurationResourceTypeNames.APIEndpointConfigurations ) .Returns(Task.FromResult(expectedServices)); @@ -133,9 +137,9 @@ public async Task GetAPIEndpointsAsync_ShouldReturnServices() // Assert Assert.Equal(expectedServices, result); - await _mockRestClient.Resources.Received(1).GetResourcesAsync>>( + await _mockRestClient.Resources.Received(1).GetResourcesAsync>>( ResourceProviderNames.FoundationaLLM_Configuration, - ConfigurationResourceTypeNames.APIEndpoints + ConfigurationResourceTypeNames.APIEndpointConfigurations ); } @@ -144,28 +148,31 @@ public async Task GetAPIEndpointAsync_ShouldReturnService() { // Arrange var serviceName = "test-service"; - var expectedService = new ResourceProviderGetResult + var expectedService = new ResourceProviderGetResult { - Resource = new APIEndpoint + Resource = new APIEndpointConfiguration { Name = serviceName, Url = "FoundationaLLM:TestAPIUrlConfiguration", - APIKeyConfigurationName = "FoundationaLLM:TestAPIKeyConfiguration", Category = APIEndpointCategory.General, - AuthenticationType = "APIKey", - APIKeyHeaderName = "FoundationaLLM:TestAPIKeyHeaderName", + AuthenticationType = AuthenticationTypes.APIKey, TimeoutSeconds = 60, - RetryStrategyName = "ExponentialBackoff" + RetryStrategyName = "ExponentialBackoff", + AuthenticationParameters = + { + { AuthenticationParametersKeys.APIKeyConfigurationName, "FoundationaLLM:TestAPIKeyConfiguration" }, + {AuthenticationParametersKeys.APIKeyHeaderName, "FoundationaLLM:TestAPIKeyHeaderName" } + } }, Actions = [], Roles = [] }; - var expectedServices = new List> { expectedService }; + var expectedServices = new List> { expectedService }; _mockRestClient.Resources - .GetResourcesAsync>>( + .GetResourcesAsync>>( ResourceProviderNames.FoundationaLLM_Configuration, - $"{ConfigurationResourceTypeNames.APIEndpoints}/{serviceName}" + $"{ConfigurationResourceTypeNames.APIEndpointConfigurations}/{serviceName}" ) .Returns(Task.FromResult(expectedServices)); @@ -174,9 +181,9 @@ public async Task GetAPIEndpointAsync_ShouldReturnService() // Assert Assert.Equal(expectedService, result); - await _mockRestClient.Resources.Received(1).GetResourcesAsync>>( + await _mockRestClient.Resources.Received(1).GetResourcesAsync>>( ResourceProviderNames.FoundationaLLM_Configuration, - $"{ConfigurationResourceTypeNames.APIEndpoints}/{serviceName}" + $"{ConfigurationResourceTypeNames.APIEndpointConfigurations}/{serviceName}" ); } @@ -186,15 +193,15 @@ public async Task GetAPIEndpointAsync_ShouldThrowException_WhenServiceNotFound() // Arrange var serviceName = "test-service"; _mockRestClient.Resources - .GetResourcesAsync>>( + .GetResourcesAsync>>( ResourceProviderNames.FoundationaLLM_Configuration, - $"{ConfigurationResourceTypeNames.APIEndpoints}/{serviceName}" + $"{ConfigurationResourceTypeNames.APIEndpointConfigurations}/{serviceName}" ) - .Returns(Task.FromResult>>(null)); + .Returns(Task.FromResult>>(null)); // Act & Assert var exception = await Assert.ThrowsAsync(() => _configurationClient.GetExternalOrchestrationServiceAsync(serviceName)); - Assert.Equal($"APIEndpoint '{serviceName}' not found.", exception.Message); + Assert.Equal($"APIEndpointConfiguration '{serviceName}' not found.", exception.Message); } [Fact] diff --git a/tests/dotnet/Orchestration.Tests/Orchestration/KnowledgeManagementOrchestrationTests.cs b/tests/dotnet/Orchestration.Tests/Orchestration/KnowledgeManagementOrchestrationTests.cs index ef151754bc..7d68ad8fd9 100644 --- a/tests/dotnet/Orchestration.Tests/Orchestration/KnowledgeManagementOrchestrationTests.cs +++ b/tests/dotnet/Orchestration.Tests/Orchestration/KnowledgeManagementOrchestrationTests.cs @@ -22,6 +22,7 @@ public KnowledgeManagementOrchestrationTests() { _knowledgeManagementOrchestration = new KnowledgeManagementOrchestration( _agent, + null, _callContext, _orchestrationService, _logger, diff --git a/tests/dotnet/Orchestration.Tests/Orchestration/OrchestrationBuilderTests.cs b/tests/dotnet/Orchestration.Tests/Orchestration/OrchestrationBuilderTests.cs index 7ffa4f8f82..87653e1285 100644 --- a/tests/dotnet/Orchestration.Tests/Orchestration/OrchestrationBuilderTests.cs +++ b/tests/dotnet/Orchestration.Tests/Orchestration/OrchestrationBuilderTests.cs @@ -35,7 +35,7 @@ public OrchestrationBuilderTests() //public async Task Build_AgentHintNotNull_KnowledgeManagementAgent() //{ // // Arrange - // var completionRequest = new CompletionRequest() + // var completionRequest = new ClientCompletionRequest() // { // UserPrompt = "Test_Userprompt", // AgentName = "knowledge-management" @@ -78,7 +78,7 @@ public OrchestrationBuilderTests() // // Act & Assert // await Assert.ThrowsAsync(async () => // await OrchestrationBuilder.Build( - // new CompletionRequest() { UserPrompt = userPrompt }, + // new ClientCompletionRequest() { UserPrompt = userPrompt }, // _callContext, // _configuration, // _resourceProviderServices, diff --git a/tests/dotnet/Orchestration.Tests/Services/SemanticKernelServiceTests.cs b/tests/dotnet/Orchestration.Tests/Services/SemanticKernelServiceTests.cs index ae8928e517..6ad14495ba 100644 --- a/tests/dotnet/Orchestration.Tests/Services/SemanticKernelServiceTests.cs +++ b/tests/dotnet/Orchestration.Tests/Services/SemanticKernelServiceTests.cs @@ -30,7 +30,8 @@ public async Task GetCompletion_Success_ReturnsCompletionResponse() // Arrange var request = new LLMCompletionRequest { - Agent = new KnowledgeManagementAgent() { Name = "Test_name", ObjectId = "Test_id", Type = "Test_type"} + Agent = new KnowledgeManagementAgent() { Name = "Test_name", ObjectId = "Test_id", Type = "Test_type" }, + UserPrompt = "" }; var responseContent = System.Text.Json.JsonSerializer.Serialize(new LLMCompletionResponse { Completion = "Completion response" }); var responseMessage = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(responseContent) }; diff --git a/tests/dotnet/SemanticKernel.Tests/Services/AzureAISearchIndexingServiceTests.cs b/tests/dotnet/SemanticKernel.Tests/Services/AzureAISearchIndexingServiceTests.cs index 77982abb64..8de530c0af 100644 --- a/tests/dotnet/SemanticKernel.Tests/Services/AzureAISearchIndexingServiceTests.cs +++ b/tests/dotnet/SemanticKernel.Tests/Services/AzureAISearchIndexingServiceTests.cs @@ -1,6 +1,7 @@ using Azure.Search.Documents.Indexes; using Azure.Search.Documents.Indexes.Models; using FoundationaLLM.Common.Authentication; +using FoundationaLLM.Common.Constants.Authentication; using FoundationaLLM.Common.Interfaces; using FoundationaLLM.Common.Models.Vectorization; using FoundationaLLM.Common.Settings; @@ -30,7 +31,7 @@ public AzureAiSearchIndexingServiceTests() new AzureAISearchIndexingServiceSettings { Endpoint = endpoint, - AuthenticationType = AzureAISearchAuthenticationTypes.AzureIdentity + AuthenticationType = AuthenticationTypes.AzureIdentity } ), LoggerFactory.Create(builder => builder.AddConsole()).CreateLogger() diff --git a/tests/dotnet/SemanticKernel.Tests/Services/SemanticKernelTextEmbeddingServiceTests.cs b/tests/dotnet/SemanticKernel.Tests/Services/SemanticKernelTextEmbeddingServiceTests.cs index 26b4bdbc68..2f3e2efd25 100644 --- a/tests/dotnet/SemanticKernel.Tests/Services/SemanticKernelTextEmbeddingServiceTests.cs +++ b/tests/dotnet/SemanticKernel.Tests/Services/SemanticKernelTextEmbeddingServiceTests.cs @@ -1,4 +1,5 @@ -using FoundationaLLM.Common.Models.Vectorization; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Models.Vectorization; using FoundationaLLM.Common.Settings; using FoundationaLLM.SemanticKernel.Core.Models.Configuration; using FoundationaLLM.SemanticKernel.Core.Services; @@ -16,7 +17,7 @@ public SemanticKernelTextEmbeddingServiceTests() _semanticKernelTextEmbeddingService = new SemanticKernelTextEmbeddingService( Options.Create( new SemanticKernelTextEmbeddingServiceSettings { - AuthenticationType = AzureOpenAIAuthenticationTypes.AzureIdentity, + AuthenticationType = AuthenticationTypes.AzureIdentity, DeploymentName = "embeddings", Endpoint = Environment.GetEnvironmentVariable("SemanticKernelTextEmbeddingServiceTestsOpenAiEndpoint") ?? "" } diff --git a/tests/dotnet/Vectorization.Tests/Services/ContentSources/DataLakeContentSourceServiceTests.cs b/tests/dotnet/Vectorization.Tests/Services/ContentSources/DataLakeContentSourceServiceTests.cs index 266782abe0..effd5617cc 100644 --- a/tests/dotnet/Vectorization.Tests/Services/ContentSources/DataLakeContentSourceServiceTests.cs +++ b/tests/dotnet/Vectorization.Tests/Services/ContentSources/DataLakeContentSourceServiceTests.cs @@ -1,4 +1,5 @@ -using FoundationaLLM.Common.Models.Configuration.Storage; +using FoundationaLLM.Common.Constants.Authentication; +using FoundationaLLM.Common.Models.Configuration.Storage; using FoundationaLLM.Common.Models.Vectorization; using FoundationaLLM.Vectorization.Services.ContentSources; using Microsoft.Extensions.Logging; @@ -12,7 +13,7 @@ public class DataLakeContentSourceServiceTests public DataLakeContentSourceServiceTests() { _dataLakeContentSourceService = new DataLakeContentSourceService( new BlobStorageServiceSettings { - AuthenticationType = BlobStorageAuthenticationTypes.ConnectionString, + AuthenticationType = AuthenticationTypes.ConnectionString, ConnectionString = Environment.GetEnvironmentVariable("DataLakeContentServiceTestsConnectionString") }, LoggerFactory.Create(builder => builder.AddConsole())