Adding subtitles to a video file

[basinbasin]

Has anyone been able to get this functionality to work? I'm able to run the following, without any errors that I can see, but the posted video does not contain subtitles. I've added some sleep commands in case there needed to be a processing pause.

#login using stored credentials
client = Client()
client.login(username, password)

videopath = r"/volume1/web/test/video.mp4"

if hasmp4 is True:
    objectpath = videopath
    vttpath = r"/volume1/web/test/subtitles.vtt"
    with open(vttpath, 'rt') as vttfile:
        vtt_data = vttfile.read()
        upload_vtt = client.com.atproto.repo.upload_blob(vtt_data)
        print("Waiting after subtitle upload...") 
        time.sleep(10)
else:
    objectpath = imagepath

# Determine locations of URLs in the post text
facets = populate_facets(postText)

with open(objectpath, 'rb') as f:
    img_data = f.read()
    upload = client.com.atproto.repo.upload_blob(img_data)
     
    print("Waiting after video upload...") 
    time.sleep(10)
    #add the video that we've uploaded    
    embed = models.AppBskyEmbedVideo.Main(video=upload.blob, alt='', captions=[
    models.AppBskyEmbedVideo.Caption(file=upload_vtt.blob, lang='en')])
                    
    time.sleep(10)
    #wrap the post send in a root_post_ref so we can use it as a reference if we want to send a reply
    root_post_ref = models.create_strong_ref(
        client.com.atproto.repo.create_record(
            models.ComAtprotoRepoCreateRecord.Data(
                repo=client.me.did,
                collection=models.ids.AppBskyFeedPost,
                record=models.AppBskyFeedPost.Record(
                    createdAt=datetime.utcnow().isoformat("T") + "Z", text=postText, embed=embed, facets=facets
                ),
            )
        )
    )

   

[MarshalX]

Did you try SRT instead of VTT? You should

[MarshalX]

client = Client()
resp = client.com.atproto.repo.get_record({'collection': 'app.bsky.feed.post', 'repo': 'did:plc:xvr3l3h5sttlahuxludxufvs', 'rkey': '3lddhqxc2zm24'})
print(resp.value.embed)

video=BlobRef(mime_type='video/mp4', size=1325499, ref=IpldLink(link='bafkreiaewwp2p46zit7fahx3rwof45qbkra3mh4psvo7srt3jv4ds3ntw4'), py_type='blob') alt='' aspect_ratio=None captions=[Caption(file=BlobRef(mime_type='*/*', size=284, ref=IpldLink(link='bafkreic2nmfqahmcg7widwuwdgtile3yq7wfbyyudlhoq5qz42amsjeuv4'), py_type='blob'), lang='en', py_type='app.bsky.embed.video#caption')] py_type='app.bsky.embed.video'

so the captions are here, right inside the record. the only suspicious thing is the mime type. which is */*

i suggest asking in discord or sniff bsky.app traffic to understand how subtitles are uploaded

[basinbasin]

Thanks for looking into this. Good shout about the mime type. Had a look at the web front end, and yes, there's a content type header of text/vtt in the blob upload.
Output for a post made through the front end has the correct mime type:

video=BlobRef(mime_type='video/mp4', size=2748232, ref=IpldLink(link='bafkreidwd7kwemzulkxjmxhk27u3kx5cdz2hds4zhbwbm5sidkyw7qa7dm'), py_type='blob') alt=None aspect_ratio=AspectRatio(height=528, width=700, py_type='app.bsky.embed.defs#aspectRatio') captions=[Caption(file=BlobRef(mime_type='text/vtt', size=271, ref=IpldLink(link='bafkreia2lngluoce5knlcydu2yudia66mx44gbxo5eneycy7bo6dcdgww4'), py_type='blob'), lang='en', py_type='app.bsky.embed.video#caption')] py_type='app.bsky.embed.video'

[MarshalX]

super duper strange that it does not set the correct mime type; typically backend guesses it based on the content; do i understand correctly that your subtitles.vtt is 100% valid VTT format?

[MarshalX]

@basinbasin try this and let me know about results pls

instead of

upload_vtt = client.com.atproto.repo.upload_blob(vtt_data)

do

response = client.invoke_procedure(
    'com.atproto.repo.uploadBlob', data=vtt_data, input_encoding='text/vtt', output_encoding='application/json'
)
upload_vtt = get_response_model(response, models.ComAtprotoRepoUploadBlob.Response)

[basinbasin]

Perfect! That's fixed it! Thank you!

Had to add in

from atproto_client.models.utils import get_response_model

But all working now.

video=BlobRef(mime_type='video/mp4', size=1879548, ref=IpldLink(link='bafkreiaq2cbjh4i3e6qe6zphriuztsrsjlzp5vhyh5pde2g76lxezureoe'), py_type='blob') alt='' aspect_ratio=None captions=[Caption(file=BlobRef(mime_type='text/vtt', size=525, ref=IpldLink(link='bafkreiergpqfxvrtzuf6bbtvdnk53fmwiqpwkazy4aoafttcxcd4kzv4xy'), py_type='blob'), lang='en', py_type='app.bsky.embed.video#caption')] py_type='app.bsky.embed.video'