The Zipf Mystery

15 sep. 2015
19 076 588 Weergaven

The of and to. A in is I. That it, for you, was with on. As have ... but be they.
RELATED LINKS AND SOURCES BELOW!
twitter.com/tweetsauce
instagram.com/electricpants
WordCount.org www.wordcount.org/
How many days have you been alive? www.beatcanvas.com/daysalive.asp
random letter generator: www.dave-reed.com/Nifty/randSeq.html
Dictionary of Obscure Sorrows: nlworld.info
Word frequency resources:
[lemmatized] en.wikipedia.org/wiki/Most_common_words_in_English
www.uow.edu.au/~dlee/corpora.htm
www.wordfrequency.info
www.anc.org/data/anc-second-release/frequency-data/
www.titania.bham.ac.uk/docs/
www.kilgarriff.co.uk/bnc-readme.html#raw
en.wiktionary.org/wiki/Wiktionary:Frequency_lists
ucrel.lancs.ac.uk/bncfreq/
[PDF] www.wordfrequency.info/files/entries.pdf
[combined Wikipedia and Gutenberg] www.monlp.com/2012/04/16/calculating-word-and-n-gram-statistics-from-a-wikipedia-corpora/
corpus.byu.edu/coca/files/100k_samples.txt
corpus.byu.edu/
corpus.leeds.ac.uk/list.html
books.google.co.uk/books?id=ja1_AAAAQBAJ&dq=word+frequency+coca&lr=
www.ling.helsinki.fi/kit/2009s/clt231/NLTK/book/ch01-LanguageProcessingAndPython.html
Great Zipf's law papers:
colala.bcs.rochester.edu/papers/piantadosi2014zipfs.pdf
www.ling.upenn.edu/~ycharles/sign708.pdf
arxiv.org/pdf/cond-mat/0412004.pdf
www-personal.umich.edu/~mejn/courses/2006/cmplxsys899/powerlaws.pdf
Zipf’s law articles and discussions:
www.theatlantic.com/magazine/archive/2002/04/seeing-around-corners/302471/
io9.com/the-mysterious-law-that-governs-the-size-of-your-city-1479244159?.Ej9760cOTJCPS_Bq4mjoww.0
plus.maths.org/content/os/latestnews/may-aug08/food/index
judson.blogs.nytimes.com/2009/05/19/math-and-the-city/?em
plus.maths.org/content/mystery-zipf?src=aop
www.datasciencecentral.com/profiles/blogs/why-zipf-s-law-explains-so-many-big-data-and-physics-phenomenons
en.wikipedia.org/wiki/Zipf%27s_law
books.google.co.uk/books?id=f8GrzlnMSm8C&pg=PA62&redir_esc=y#v=onepage&q&f=false
arxiv.org/pdf/0802.4393v1.pdf
www.pnas.org/content/108/9/3526.full
lewisdartnell.com/language_page.htm
wugology.com/zipfs-law/
other Zipf’s law PDFs
ftp.iza.org/dp3928.pdf
arxiv.org/pdf/1402.2965.pdf
arxiv.org/pdf/1104.3199.pdf
www.lel.ed.ac.uk/~jim/zipfjrh.pdf
www.ncbi.nlm.nih.gov/pmc/articles/PMC2834740/#pone.0009411-Mandelbrot1
polymer.bu.edu/hes/articles/pgs02a.pdf
in untranslated language: arxiv.org/pdf/0808.2904.pdf
pages.stern.nyu.edu/~xgabaix/papers/zipf.pdf
www.hpl.hp.com/research/idl/papers/ranking/ranking.html
statweb.stanford.edu/~owen/courses/306a/ZipfAndGutenberg.pdf
arxiv.org/pdf/1310.0448v3.pdf
www.kornai.com/Papers/glotto5.pdf
Zipf’s law slides:
www.slideshare.net/guest9fc47a/nlp-new-words
Pareto Principle and related ‘laws’:
www.squawkpoint.com/2013/03/pareto-principle/
billyshall.com/blog/post/paretos-principle
en.wikipedia.org/wiki/Pareto_principle
Random typing and Zipf:
www.longtail.com/the_long_tail/2006/09/is_zipfs_law_ju.html
health 80/20: archive.ahrq.gov/research/findings/factsheets/costs/expriach/expriach1.html
Principle of least effort:
en.wikipedia.org/wiki/Principle_of_least_effort
en.wikipedia.org/wiki/Satisficing
www.pnas.org/content/100/3/788.full.pdf [PDF]
csiss.org/classics/content/99
self organized criticality:
journal.frontiersin.org/article/10.3389/fnsys.2014.00166/full
Hapax Legomenon:
campus.albion.edu/english/2011/02/15/hapax-legomenon/
www.dailywritingtips.com/is-that-a-hapax-legomenon/
en.wikipedia.org/wiki/Hapax_legomenon
[PDF] www.aclweb.org/anthology/J10-4003
www.wired.com/2012/01/hapax-legomena-and-zipfs-law/
oed.hertford.ox.ac.uk/main/content/view/402/450/index.html#_ftn1
oed.hertford.ox.ac.uk/main/content/view/36/166/index.html
Learning curve: en.wikipedia.org/wiki/Learning_curve
Forgetting curve:
www.trainingindustry.com/wiki/entries/forgetting-curve.aspx
en.wikipedia.org/wiki/Forgetting_curve
Experience curve effects: en.wikipedia.org/wiki/Experience_curve_effects
Forgetting
and zipf's law: act-r.psy.cmu.edu/wordpress/wp-content/uploads/2012/12/37JRA_LS_PS_1991.pdf
public.psych.iastate.edu/shacarp/Wixted_Carpenter_2007.pdf
marshalljonesjr.com/youll-remember-less-than-001-of-your-life/
en.wikipedia.org/wiki/Forgetting
www.reddit.com/r/Showerthoughts/comments/3gu9qk/it_only_takes_three_generations_for_you_to_be/
music from:
nlworld.info
www.audionetwork.com

Reacties
  • Maybe it's like placing tiles in a bathroom, most of the surface will be covered by more basic shapes that fit together in all sorts of ways. However, each bathroom is different and sometimes you need a more specific cut but ever so less than the basic shape like a square or a triangle which you use primarily.

    Uber LauferUber Laufer2 uur geleden
  • Yea but how much does here weight?

    Tommaso cognomeTommaso cognome3 uur geleden
  • brain small. math complicated.. there is no actual mystery.

    Tom Beats BeatsTom Beats Beats5 uur geleden
  • The I that is on it was of it and in it to you, for with it they be but a that is with and was as they have but be...

    UserUser10 uur geleden
  • George Zipp was also Ted Strykers wing man in the air battle over Macho Grande. He didn't made it.

    Andrews Charles BloyAndrews Charles Bloy11 uur geleden
  • S

    Chanman176 _Chanman176 _12 uur geleden
  • T H E

    Polter ObtoraticPolter Obtoratic13 uur geleden
  • "Quizzaciously" may not appear much in writing, but is captured in an extremely popular and widespread meme of a cross-eyed aquatic sponge bending over a long ways with his hands on his hips, repeating back whatever was previously said in mixed capitalization.

    KosquoKosquo15 uur geleden
  • What is the ending song?

    DAVASDAVAS21 uur geleden
  • mumbo jumbo !!??

    ItsMe NathanItsMe NathanDag geleden
  • Alguém mais lembrou desse vídeo depois de fazer o ENEM?

    Douglas RauberDouglas RauberDag geleden
  • "The Rich get richer. Its just math." No it isn't. Its Capitalism.

    Das TIERDas TIERDag geleden
  • What mumbo jumbo is doin here

    Mantvydas KazakeviciusMantvydas KazakeviciusDag geleden
  • Everything has a 80% ≥ 20% it's kinda weird...

    SaedrisSaedrisDag geleden
  • me when the. when the

    KillerQueenKillerQueenDag geleden
  • the

    TimeSFTimeSFDag geleden
  • I'll simplify this video for you. 🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️

    Shorey BanksShorey BanksDag geleden
  • I've been being recommended this for years and I'm finally bored enough to watch it

    BjornHubBjornHubDag geleden
  • Just convinced me we are definitely in a simulation

    17Canine1717Canine17Dag geleden
  • It was fun to see that you said “the” twice as often as “of”.

    John JohnsonJohn JohnsonDag geleden
  • sauce

    CodyCodyDag geleden
  • I think it goes back to the brain itself and the physics behind how consciousness is encoded...we have no choices but to ultimately produce this sequence. Maybe. Admittedly, first I heard of this.

    Vikram GuptaVikram GuptaDag geleden
  • They got lazy with the simulation...

    Jonah 2007Jonah 2007Dag geleden
  • One of the best.

    LordVoldemort's sidechickLordVoldemort's sidechickDag geleden
  • The soggy t-shirt reilly drain because description spontaneously travel on a aquatic study. acrid, dull tax

    Jacob CassarJacob CassarDag geleden
  • WTF

    garfieldgarfield2 dagen geleden
  • the example in 18:30 probably shows more how's Zipf's law is "our brain".. but actually there are people who actually can remember each day of their lives. However, now I realize that the number of people that remember only few days of their lives and the number of people that remember most days of their lives, also obeys Zipf's law. Damn, it's undeniable!!! xD

    KFoxInKFoxIn2 dagen geleden
  • Hey Michael, long time follewer here! I love the watch you are wearing in this video, can you tell me which brand and model it is? You’ll probably won’t see this comment but I’ll try my chances. Thanks in advance!

    Turan MüsevitoğluTuran Müsevitoğlu2 dagen geleden
  • Zipfian = the accumulation of advantage in language? Interesting.

    BeyondfubarBeyondfubar2 dagen geleden
  • 80% of the views come from 20% of subscribers...20% of people in your life are the real ones

    Dumbass ProductionsDumbass Productions2 dagen geleden
  • When all of the comments use the word "the".

    United Armed Forces ConglomerateUnited Armed Forces Conglomerate2 dagen geleden
  • I don't get calls

    Billu #mnbvcBillu #mnbvc2 dagen geleden
  • Tinder's law: 1% of males get 100% of women

    Piotr BPiotr B2 dagen geleden
  • I don't know who you are or where you came from but I found this channel 2 days ago and now I'm crossed eyed.

    BroharnBroharn2 dagen geleden
  • 18:48 Whoa whoa whoa whoa! Michael's birthday is on January 23rd? How did I just happen to receive my first Curiosity Box, and feel inclined to go back and re-watch a bunch of Vsauce videos on Michael's birthday?

    MoolaMoola2 dagen geleden
  • Me: "I don't remember most of what I read in books". Also me, 20 years after reading A Wrinkle In Time: "C is the square of the velocity of light in centimeters per second."

    Kazzy and DecchanKazzy and Decchan2 dagen geleden
  • Mind blown.

    Jane AJane A2 dagen geleden
  • I’m guessing 20% of the NLworld channels have 80% of the total views on NLworld?

    David WilliamsDavid Williams2 dagen geleden
  • Great RW Emerson quote.

    Len PalmeriLen Palmeri2 dagen geleden
  • kdasj fkhdsi; kdnwejf; saijf sdmfjsda; fjkl huijosd hfjmsbfjksl i zfkjmje snj fhgeuj fusiod fkjmsd hjfsd fjksdl; fjks bfnjs bfvgudsxfv usdfzjk dns mklf hesdujvfifsd zkjfl;dhsjk mZfbjsl fujsd fjsdm hzfjdlhv iujsd fjmsd ;b jffls duvjsd jfghsdjv l sjkfhsd j FBvhdjsl VHfhdsLK F This can't be Zipfy. After all, I have only 2 thumbs to hit the space bar with, and 8 fingers to hit other letters with. So I am actually more likely to have short words, but VERY short words are rather unlikely due to the time it would take to use a thumb twice rather than use any of the other four fingers instead of the second thumb stroke. Also 16:15 I'm not sure that note is legal. If we are talking about words, I would expect to talk about _each unique word manifestation_ . Otherwise you are just defining the rank of certain types of words, like "article adjectives are the most common, prepositions are the second most, conjuntions are the third most," rather than the words themselves. That is just an illustration of how human communication trends are, which is obviously independent of language itself, because humans all conceptualize and use language the same way. "Say the most content, with a limited density, a maximum simplicity, and the shortest amount of time/syllables possible". And that's just Dijkstra's algorithm, with a few modifications I'm sure. Quizzaciously is just a hard word to say, and it's long. So, of course we wouldn't use it. If anything, Zipf-ian systems are just systems of priority, like Dijkstra's algorithm. You pick the favorable quality, and you ignore the rest because you don't have the processing power to keep it all. You forget the mundane, but all things deemed relevant or important are remembered. And occasionally, random facts are also randomly selected to be "included". If I'm a hiring manager, and I want to hire someone with X, Y, and Z qualities, then whatever resumes I see with X, Y, and Z will get the most of my time, and those without one or more of those desirable traits will receive very little, if any, of my time. It's just a prioritization system. A naturally occurring prioritization system.

    Hayato TenryuuHayato Tenryuu3 dagen geleden
  • 80% of the battery usage is by 20% of the processes.

    Andrew QiAndrew Qi3 dagen geleden
  • sauce

    Ikenna OkonkwoIkenna Okonkwo3 dagen geleden
  • Was "the" a pun?

    Dead Guy Dude ManDead Guy Dude Man4 dagen geleden
  • I wonder what a book that was intentionally written to not follow this would be like?

    Jacob RogersJacob Rogers4 dagen geleden
  • Wordcount uses Flash. And they haven't updated the site.

    Jonah FalconJonah Falcon4 dagen geleden
  • Isn't this mathematically normal? the total odds/frequency of anything will be 1 and when ranked by order in a graph will appear the same way? otherwise the total odds will either exceed or never reach 1, which can't happen.

    Aladen ZannerniAladen Zannerni4 dagen geleden
  • The most thought word must be "F" word though,🤔

    Prakhar SharmaPrakhar Sharma4 dagen geleden
  • I think the most used Hungarian word is "B@zdmeg" (i needed to censor it). It's a swear word. And it's beautiful.

    Bodor JánosBodor János4 dagen geleden
  • "Quizzaciously" is more popular now. There is an article in Urban Dictionary.

    Dron008Dron0084 dagen geleden
  • duh!?

    Somenath GaraiSomenath Garai4 dagen geleden
  • wait...so does this mean 20% of all youtubers are actually making videos and are popular while 80% are only watchers ? does this means 20% out of those who make videos in youtube are popular and 80% are barely known ? does this mean 20% out of all of the comments in a video gets popular while 80% just fly by and barely get over 5 likes ? ...does this mean 20% out of everyone in this world...get rich while 80% just die poor does this mean 20% of people in this world die alone while 80% the opposite ? 20 80 rule is...scary and weird tbh it's just way too weird to think about it...this thing really did caught my attention

    kiro cazamkiro cazam4 dagen geleden
  • Any German here? I Like the Title😂

    aabicaabic4 dagen geleden
  • Ancient language that hasn't been translated yet-

    Anex AntonyAnex Antony5 dagen geleden
  • Wth is this wizardry! I was just thinking this morning about this vsauce video I’d seen years ago but I couldn’t remember what it was called! I only remembered the final quote from Emerson. And NLworld just suggest this to me today. 🤯

    5MinPhil5MinPhil5 dagen geleden
  • Vsauce: sauce is the 5,555 th mosts common english word Also Vsauce every video: 0:00

    Doby AdobyDoby Adoby5 dagen geleden
  • The 45° angle line we get when graphing the frequency of words is not just for humans. The different sounds made by Dolfin show this 45° angle pattern as well.

    UndefindUndefind5 dagen geleden
  • 80% of my brain is fried by the first 20% of this video

    Dish Soap GamingDish Soap Gaming5 dagen geleden
  • Idk if you're reading comments but what do you think, would a auto suggest feature on keyboard follow zipfs law. If we humans follow his law everywhere what's the possibility of computer with auto suggest for words doing the same? (i mean there is chance it would because we created it)

    N!ckyN!cky5 dagen geleden
  • So, flipping someone the bird while driving is responsible for about 95% of the road rage that occurs. Makes sense...

    Zach SchmiedZach Schmied5 dagen geleden
  • Most things go Top to bottom, you don't start a cake with a "hint of cinnamon " or " a cherry" even someone doing there accounts would start with big number first.

    Robcop WesttonRobcop Westton5 dagen geleden
  • would have been funny if he didnt say the for the entire video

    Pigbelly19 ‘Pigbelly19 ‘5 dagen geleden
  • mfw i suddenly realise the background music at 12:45 is actually bach double but jaunty piano mode

    Jacob CostenJacob Costen5 dagen geleden
  • Yeah its simple.

    ِِ5 dagen geleden
  • Where did I put my paper clips?

    Toms MarkovsToms Markovs5 dagen geleden
  • Answer next tk questions

    ِِ5 dagen geleden
  • A similar law is Benford's Law.

    Vineet SharmaVineet Sharma5 dagen geleden
  • me and the bois going on to wikapedia and making scauce the 69 most used word

    Henry AndersonHenry Anderson5 dagen geleden
  • Reading up on superspreading events and epidemiology, the 80/20 rule was also observed -- 80% of disease transmission is caused by 20% of infected cases

    TheMockingJayTheMockingJay6 dagen geleden
  • there is no way "school" is uses more than "no"

    Cameron GoodallCameron Goodall6 dagen geleden
  • The Bible was written by man bc it follows ziffs law

    blocboy jbblocboy jb6 dagen geleden
  • we bouta get quizzacious

    Matt GarreltsMatt Garrelts6 dagen geleden
  • This is Benford's law

    Andrej AjnikAndrej Ajnik6 dagen geleden
  • The absent reason putatively guide because transaction regretfully milk besides a overrated territory. aboard, tranquil skirt

    Andrew ZeitlerAndrew Zeitler6 dagen geleden
  • Sounds very similar to Benford's law also. There seems to be a more primary factor causing the occurrence of this throughout various systems. Living in a simulation becomes more plausible the more you realize the patterns.

    Dennis GarageDennis Garage7 dagen geleden
  • I think this all comes down to usefulness and meaning. The could be used to pinpoint any other word while sauce is so specific that it isn’t used as much because the explanation is direct and intentional. The is what exactly? but sauce is sauce.

    Lu BuLu Bu7 dagen geleden
  • 80% of my stories are filled with um and 20% of my stories are actually told

    ABC DEFABC DEF7 dagen geleden
  • Dude, after 2 video spirals on your channel I'm not ashamed to admit that although chunks of the content my idiot brain has no way to comprehend it. But the way you present and explain the info, what I end up being able to fathom, blows my mind! Love your content.

    John HeggertyJohn Heggerty7 dagen geleden
  • Bruh

    Shard FloofFoxShard FloofFox7 dagen geleden
  • We live in a simulation.

    Drooby Dooby DooDrooby Dooby Doo7 dagen geleden
  • Who else tried to count the amount of times he said "the"

    Tuckster04Tuckster047 dagen geleden
  • This 80/20 split forecasted Tfues contract

    Edward TrevinoEdward Trevino7 dagen geleden
  • It indicates that there is one creator of the universe.. The One and Only God

    Aljulanda AlmaghadariAljulanda Almaghadari7 dagen geleden
  • 3:03 wow

    Kjonesey ‘Kjonesey ‘7 dagen geleden
  • 80 percent of my test is about 20 percent of what we had to study

    Elise VisserElise Visser7 dagen geleden
  • Meanwhile: German *der die das die den die das die dem der dem den des der des der*

    a aa a7 dagen geleden
  • since this cideo was published, Quizzaciously has gone from 1 result, to 17,500 results. Proving your point that the more something is used, the more it be used again.

    Ryan CooperRyan Cooper8 dagen geleden
  • 2:08 normal people: wow people really like sauce. me being a weeb: ( ͡° ͜ʖ ͡°)

    Nooby-kunNooby-kun8 dagen geleden
  • Zipf's Law is just the algorithm that God put into a lot of shit after a long day of developing the " everything"

    Cedrick NarsisoCedrick Narsiso8 dagen geleden
  • Vsauce always makes me feel philosophical

    toastboiitoastboii8 dagen geleden
  • my grandma did a funny dance on the floor and i dont know why vsauce help

    EE8 dagen geleden
  • in the beginning was the word, and the word was with God and the word was God

    papinbalapapinbala8 dagen geleden
  • Is this proof that we’re in a simulation?

    James TaylorJames Taylor8 dagen geleden
  • This law explains why you can never solve the problem of inequality in every single measurement of socio-economic status.

    pizzasmithdudepizzasmithdude8 dagen geleden
  • I think that Zipf was correct. Speaking more than ten languages myself, I've come to the theory that language is just a way to transfer thoughts. In doing so you will need a (rare) specific word and several (common) words to describe how people should interpret the specific word.

    Mark PrinsMark Prins8 dagen geleden
  • "[...] in *the* english language. About one in every 16 words you encounter on a daily basis is *the* ." son of a...

    Lady YmirLady Ymir8 dagen geleden
  • Hapax Legomenon derives from the Greek Άπαξ λεγόμενον which literally means “ said once”.

    Deconstruct//RecreateDeconstruct//Recreate8 dagen geleden
  • so then what happens when 99% of the wealth is with 1% of the people. such a huge imbalance im sure Cthulu will come and have to balance that out. or maybe thats why the sun is freaking the fuck out and is about to spit a micro nova at us.

    Brave CatBrave Cat8 dagen geleden
  • michael saying "80 percent and..." so much fucked me over ngl

    KUWALDAKUWALDA9 dagen geleden
  • It is funny how the Pareto principle is relevant today. Here in Quebec, Canada, they said that 20% of the people at risk of contracting the COVID-19 virus (mostly people over 65 years old) represent 80% of the people hospitalized in our hospitals! I didn't know about that principle then!

    AkylliaAkyllia9 dagen geleden
  • too bad i can't check out wordcount since flash player is dead

    QwertyQwerty9 dagen geleden
NLworld