Google recently launched "SeektoAction" as an addition to the Video Schema, however, what URL you add is not clear.
Here is their example:
"@type": "SeekToAction",
"target": "https://video.example.com/watch/videoID?t={seek_to_second_number}",
By "video.example" do they mean to add the page on your site or the video link either Vimeo, Youtube or something else?