Computers just got a lot better at writing

Computers just got a lot better at writing

How machines can mimic our language.

Join the Open Sourced Reporting Network: http://www.vox.com/opensourcednetwork

Something big happened in the past year: Researchers created computer programs that can write long passages of coherent, original text.

Language models like GPT-2, Grover, and CTRL create text passages that seem written by someone fluent in the language, but not in the truth. That AI field, Natural Language Processing (NLP), didn’t exactly set out to create a fake news machine. Rather, it’s the byproduct of a line of research into massive pretrained language models: Machine learning programs that store vast statistical maps of how we use our language. So far, the technology’s creative uses seem to outnumber its malicious ones. But it’s not difficult to imagine how these text-fakes could cause harm, especially as these models become widely shared and deployable by anyone with basic know-how. Read more here: https://www.vox.com/recode/2020/3/4/21163743/ai-language-generation-fake-text-gpt2

Open Sourced is a year-long reporting project from Recode by Vox that goes deep into the closed ecosystems of data, privacy, algorithms, and artificial intelligence. Learn more at http://www.vox.com/opensourced

This project is made possible by the Omidyar Network. All Open Sourced content is editorially independent and produced by our journalists.

Watch all episodes of Open Sourced right here on YouTube: http://bit.ly/2tIHftD

Try out natural language generation and detection with these tools:
https://demo.allennlp.org/next-token-lm
https://talktotransformer.com/
https://transformer.huggingface.co/
https://grover.allenai.org/
https://www.ai21.com/haim
http://gltr.io/
https://play.aidungeon.io/
https://huggingface.co/openai-detector/

Sources:
https://ruder.io/nlp-imagenet/
https://medium.com/@ageitgey/deepfaking-the-news-with-nlp-and-transformer-models-5e057ebd697d
https://openai.com/blog/better-language-models/
https://blog.einstein.ai/introducing-a-conditional-transformer-language-model-for-controllable-generation/
https://veredshwartz.blogspot.com/2019/08/text-generation.html
http://www.mattkenney.me/gpt-2-345/
http://www.mattkenney.me/gpt-2/
https://jalammar.github.io/illustrated-gpt2/
https://mc.ai/introduction-to-language-modelling-and-deep-neural-network-based-text-generation/
https://fortune.com/2020/01/20/natural-language-processing-business/
https://www.vox.com/future-perfect/2019/2/14/18222270/artificial-intelligence-open-ai-natural-language-processing
https://www.newyorker.com/magazine/2019/10/14/can-a-machine-learn-to-write-for-the-new-yorker
https://www.youtube.com/watch?v=GEtbD6pqTTE
https://arxiv.org/pdf/1905.12616.pdf
https://arxiv.org/abs/1911.03343
https://arxiv.org/abs/1904.09751
https://techscience.org/a/2019121801/
https://www.middlebury.edu/institute/sites/www.middlebury.edu.institute/files/2019-11/The%20Industrialization%20of%20Terrorist%20Propaganda%20-%20CTEC.pdf?fv=TzdJnlDw
http://newsyoucantuse.com/
https://aiweirdness.com/post/168051907512/the-first-line-of-a-novel-by-an-improved-neural
https://aiweirdness.com/post/159302925452/the-neural-network-generated-pickup-lines-that-are
https://www.nytimes.com/interactive/2018/10/26/opinion/halloween-spooky-costumes-machine-learning-generator.html
https://aiweirdness.com/post/160985569682/paint-colors-designed-by-neural-network-part-2
https://www.reddit.com/r/SubSimulatorGPT2/

https://cloud.google.com/text-to-speech/

Vox.com is a news website that helps you cut through the noise and understand what’s really driving the events in the headlines. Check out http://www.vox.com.

Watch our full video catalog: http://goo.gl/IZONyE
Follow Vox on Facebook: http://goo.gl/U2g06o
Or Twitter: http://goo.gl/XFrZ5H

50 Comments

  1. PX Xaver on February 18, 2021 at 10:04 pm

    My kids are probably gonna have friends that don’t even exist and are just computer generated accounts on social media or something.



  2. Tasman Millen on February 18, 2021 at 10:04 pm

    Plot twist: This whole video was written by an AI

    And so was this comment



  3. Liam John on February 18, 2021 at 10:07 pm

    Hello



  4. AfonsoSantos on February 18, 2021 at 10:07 pm

    Trust me, in some years, our world is gonna be exactly like sci fi movies where you can’t tell who’s a real human or not.



  5. S_AME on February 18, 2021 at 10:08 pm

    Swift Keyboard on android does this and it’s really helpful.



  6. TheWalkingBritt on February 18, 2021 at 10:10 pm

    In five years of service and a new song from a local community in which the world is the best of the most beautiful woman in your life with you guys are the most popular in the morning and happy to be the same thing as the new album.



  7. Vox on February 18, 2021 at 10:12 pm

    To learn more about this technology, and try it out yourself, check out the links in the description box. Thanks for watching! -joss



  8. Ale Chacon on February 18, 2021 at 10:12 pm

    Well that got dark and then just started talking about it hahah
    -my iPad 2020



  9. Josh on February 18, 2021 at 10:13 pm

    Natural Language Processing!



  10. wala Ogele on February 18, 2021 at 10:13 pm

    I am looking for a while. I have a video of the most important thing is that the information contained in this email address. Please note that the information contained in this email address. Please note that the new job is to be the best way to the UK and



  11. Liam John on February 18, 2021 at 10:15 pm

    S



  12. Rob Fowler on February 18, 2021 at 10:16 pm

    Imagine the day when a language AI creates a religion and then all of Scientology falls for it.



  13. TallMigrant on February 18, 2021 at 10:19 pm

    It is not the only reason smh that I am a man with no idea of what he has done tomorrow and he will be a good man with the same square and I am not going with him for the first time



  14. Jonathan Ross on February 18, 2021 at 10:20 pm

    I love the weird news stories from Vox.



  15. Bafflin Book on February 18, 2021 at 10:20 pm

    In five years time I will also have a smooth transition to the first part of the shrimp project with my current organisation and I have medical staff flirting with each other 🤣🤣



  16. p s on February 18, 2021 at 10:21 pm

    somewhere software engineer guys /gals are working on random youtube inspirational vid generator ,by studying VOX vids



  17. IndieHustle on February 18, 2021 at 10:22 pm

    Brilliant. These bots are already spamming things like comments from the New York Times, but also hard right sites as well. Making racist, xenophobic, and fringe opinions seem casual….we already did that..nothing to see here folks..



  18. Player Review on February 18, 2021 at 10:24 pm

    deconstructionism/post-structuralism can sometimes manifest just as poorly, but that is the inherent fallability of a human author. A programs failure is from its programmer. No escaping the flawed nature of data journalism when the subjective composes it for the subjective consumer; Implied reception and filtered inferrence. In the end, we decide what meets our respective standards, perhaps crawling the perspectives and parsing out the lost nuance – Still an educated guessing game in instances outside the absolute sciences.



  19. Leon Leb on February 18, 2021 at 10:24 pm

    Message with language model:
    Is the best way for you to understand how you can do anything to help your child succeed in the future and how to get gigantamax pokémon



  20. Przemek Pawlikowski on February 18, 2021 at 10:26 pm

    Out of topic. Who is the author of the last piece of music in this material? What is the title of this music? Regards



  21. kcharris2 on February 18, 2021 at 10:26 pm

    "It was a rainy, drizzling day in the summer of 1969. And the people of new york, who had become accustomed to the warm, kissable air of the city, were having another bad one."



  22. Praveen Nirmalkar on February 18, 2021 at 10:28 pm

    This is scary. We need to stop it.



  23. Prabhjot Singh on February 18, 2021 at 10:29 pm

    I’m a writer, and after watching this, I’m seriously reconsidering my line of work.



  24. SpilledSprite on February 18, 2021 at 10:30 pm

    And how do I know vox isn’t a robot?



  25. pbjtoast on February 18, 2021 at 10:32 pm

    This video is a lot of the time I was in school and hosted the group project meetings for research and I looked at the shop on Sunday night and I looked at the shop on Sunday night and I looked at the shop on Sunday night and

    Yea, I haven’t had mine turned on in forever because of how eerie it is, and now it doesn’t work that well (I just typed "this video" and the rest of the sentence was predictive)



  26. Theodore McComiskey on February 18, 2021 at 10:32 pm

    I am the one that likes to be a white and white white and blue white and white white and the white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white white gold white white and white.



  27. M Brontë on February 18, 2021 at 10:33 pm

    A.I. linguistics is still in grade 4….just wait till it reaches grade 10 and starts selling dope.



  28. Fawn on February 18, 2021 at 10:34 pm

    Just send me a picture when you’re done with the show and then you can send me the message that she sent me and then send me a text



  29. Andrew Marmorstein on February 18, 2021 at 10:36 pm

    Breaking: This video was narrated by a bot.



  30. Conner Fields on February 18, 2021 at 10:36 pm

    It’s like the computer’s attempt at automatic writing (that early-surrealist literary technique).



  31. TheWolfX on February 18, 2021 at 10:40 pm

    I think computers got mad when we called them bots at writing stories, so they got better.



  32. odd . rolls on February 18, 2021 at 10:43 pm

    Did anyone else realize that the very first autocorrect was like an Overwatch fanfiction?



  33. TheWalkingBritt on February 18, 2021 at 10:45 pm

    All I can do to attempt to avoid bots on Twitter is to avoid people with generic Firstname-lastname-numbers usernames. For example ericbooker5677740 or jessicacollins478357. Not fool proof but it’s all I got



  34. Dattatreya Chakraborty on February 18, 2021 at 10:46 pm

    Next technological breakthrough we need: finding the cure for Corona, world hunger, cancer.
    Capitalist machinery: Nah let’s take the job from writers.



  35. Sapien on February 18, 2021 at 10:47 pm

    I am a Mask right now because I have ghost of all I can😂🤣😭😂🤣😭😂🤣😭😂😂🤣😭😭😩😭😩😂😂

    Scary suggest words 😱



  36. Yi Ren on February 18, 2021 at 10:48 pm

    I see. So news companies are actually AI giants.



  37. Mariah306 on February 18, 2021 at 10:48 pm

    This video went from informative to scary in 0.5 seconds. Mind Blown!



  38. Meitemarkiano on February 18, 2021 at 10:49 pm

    I’m not sure what about you guys I think you should just use your turn on your way to work I don’t think I can do anything to make you feel better but I’m gonna I was just gonna was the sun and the moon sun sun to get to you know you have the snow you and I wanna was a delish dish you made it in your way to your house so you know how much you have been to eat and eat it all the day and you get to see the kids in your life you can get them and get to eat the food is so much better and then I don’t think you have a good job and I wanna is a way you can eat and a little bit of rest but I wanna was the way you to get a little more snow then I don’t think I can eat anything and eat it and I don’t have a lot to do so that’s what I’m doing now I cannot is a way I can get a chicken or something like pizza and a type a a chicken and IRN BRU was the way you eat and I don’t have to go eat anything I eat chicken or something or chicken or something haha was a good morning and then you have the time you get here you and you can get it and get some chicken 🍗 was a chicken and IRN was the day we had to go eat and a cow 🐮 I cannot is a chicken and chicken 🐔 I just got off my Twitter I wanna get you some good afternoon 😃 was the day you get to eat and get a drink 🥃 here was



  39. Aiden Whitaker on February 18, 2021 at 10:49 pm

    okay you horrific pig of a Democrat. It really shows how bright you and everyone else is, watching and believing all that vox has to say. The only truth to vox is that they make cool looking vids. Other than that it’s just radical left ideas- trying to be normalized. Don’t believe me? Maybe you should question their "dont ask dont tell" mentality of where their facts come from, and look at the inaccuracies in their graphs/tables/charts/ect. They talk about how the right is radical, yet we just believe in the constitution and a somewhat democratic state. (Look at the electoral college. It exists to give small states a chance at electing who they want. *Aka the minorities that dems care oh so much about*)



  40. Sujal Gurung on February 18, 2021 at 10:51 pm

    I’m just wondering about how close we are to Roald Dahl’s The Great Automatic Grammatizator becoming a reality.



  41. Supreme Xdee on February 18, 2021 at 10:52 pm

    Make a better title. this is bait



  42. TheWalkingBritt on February 18, 2021 at 10:54 pm

    All I can do to attempt to avoid bots on Twitter is to avoid people with generic Firstname-lastname-numbers usernames. For example ericbooker5677740 or jessicacollins478357. Not fool proof but it’s all I got



  43. rollership on February 18, 2021 at 10:55 pm

    which of these youtube comments are real..



  44. Warrcoww on February 18, 2021 at 10:56 pm

    Time to roll out the ol’ Voight Kampff machine



  45. Eduardo Rodriguez on February 18, 2021 at 10:57 pm

    What is the main idea of this video. For a summary.



  46. Tristan on February 18, 2021 at 10:59 pm

    they should make a black mirror episode



  47. Caffeinated Nation on February 18, 2021 at 10:59 pm

    *Black Mirror*



  48. tomme le homme on February 18, 2021 at 10:59 pm

    I am a piece of shot. That is what I said
    Turned into

    I am a piece of shot of this invention to provide a method and system and a lot of people are going to get it right away from the city center of the room with a lot more of the same, and I a little more time in a way ↕️ to be able to do that and the rest 😪 to be able to do this for the my life is to



  49. jigyanshu shrivastava on February 18, 2021 at 11:00 pm

    A.I can not understand things.



  50. Bhavneet Singh Bajwa on February 18, 2021 at 11:02 pm

    Ending Music: amazing… source?