“Deepfakes” of audio and video have made a lot of news lately, with stories making the including a fake video of Mark Zuckerberg that looks lifelike and scam phone calls being made to major companies that sound exactly like their CEOs.
These are powered by LSTM cells, or “Long Short Term Memory” neural networks, which are capable of learning from a set of given input data and then using their memory to predict (or fake) future events. These cells have been extremely powerful in many domains including data science, but tend to be hampered by the fact that they rely on the time step between data to be uniform (e.g. video where it is always 24 pictures per second)
Below are collected thoughts on a new, modified version of LSTM cells called T-LSTM’s that are more capable of understanding data where the time step between events is variable, such as in healthcare when doctor’s visits can be grouped close together or spaced further apart.
These modified cells have proven to be very effective so far, and could be used more widely to great effect in the future.
https://github.com/mkrupczak3/T-LSTM
Related:
