add document for GenAI #7170
add document for GenAI #7170
Conversation
|
@NiklasGustafsson @stephentoub @SteveSandersonMS can you please take a look and provide feedback. This is the design document for the GenAI library which consolidates the models ported to TorchSharp (Phi, Llama, StableDiffusion) into a single library for easier consumption. |
luisquintanilla
requested review from
SteveSandersonMS,
NiklasGustafsson and
stephentoub
| { | ||
| public virtual ( | ||
| Tensor, // output token ids [batch_size, sequence_length] | ||
| Tensor // output logits [batch_size, sequence_length, vocab_size] |
There was a problem hiding this comment.
Is this the TorchSharp tensor type? Can / should it become the System.Numerics.Tensors one?
There was a problem hiding this comment.
Is this the TorchSharp tensor type
Yes
should it become the System.Numerics.Tensors one?
If System.Numerics.Tensors can work with libtorch, then maybe yes.
There was a problem hiding this comment.
We're some ways from having Tensor integrated with TorchSharp, it is still experimental and subject to change.
There was a problem hiding this comment.
And it would be the TorchSharp shim implementing ITensor, not Tensor from SNT.
There was a problem hiding this comment.
Presumably it would be ITensor<T> such that a Tensor<T> could be passed in? The implementation could type test if it were a TorchSharp implementation and act accordingly, presumably.
There was a problem hiding this comment.
Yes, in the fullness of time.
| where TTokenizer : ITokenizer | ||
| where TCausalLM : nn.Module<CausalLanguageModelInput, CausalLanguageModelOutput> | ||
| { | ||
| public CausalLMPipeline<LLama2Tokenizer, Phi3ForCasualLM> Create(LLama2Tokenizer tokenizer, Phi3ForCasualLM model); |
There was a problem hiding this comment.
Presumably this is intended to instead be a static method on the base class?
There was a problem hiding this comment.
Hmmm on the second thought, maybe not. Because CasualLMPipeline and CasualLMPipeline<TTokenizer, TCasualLM> will live in GenAI.Core while Phi3ForCasualLM lives in GenAI.Phi3.
In that case, this Create method is no longer necessary exist, using a constructor to take TTokenzier and TCausalLM would be easier to understand
We are excited to review your PR.
So we can do the best job, please check:
Fixes #nnnnin your description to cause GitHub to automatically close the issue(s) when your PR is merged.Add desgin document for Add GenAI packages #7169