去投票

AI isn't gonna keep improving – YouTube Dictation Transcript & Vocabulary

歡迎來到 FluentDictation,最佳的 YouTube 聽寫網站。使用我們的互動逐字稿及跟讀工具,掌握此 B1 影片。"AI isn't gonna keep improving" 已被拆分為易於練習的片段,適合聽寫與發音提升。閱讀標註逐字稿,學習關鍵詞彙,增強聽力技巧。 👉 開始聽寫練習

加入數千學習者,使用 YouTube 聽寫工具提升英語聽力與寫作技能。

📺 Click to play this educational video. Best viewed with captions enabled for dictation practice.

互動逐字稿與重點

1.I got a Hut take for y'all today I think we might have hit an AI Plateau well we haven't hit it yet I think we're getting there fast what do I mean this can't be possible right when we look at all the new models and all the crazy things you can do with them the improvements from chat GPT 2 to 3 to 3.5 to four with Claude coming out of nowhere and being really good with open source models like llama and mistol we can't be out a plateau that's crazy right well there's a lot of things that have these patterns and I want to start with with a bit of an interesting tangent I want to talk about Mo's law if you're not familiar Mo's law is an old concept from the programming world it's a law created it's not a real law it was some from a Dev and a cardware back in the like what' 7s and he noticed how fast things were improving in terms of performance his is that the number of transistors on a microchip roughly doubles every 2 years and the cost is haved over the same time time frame so if you had a chip let's say that had four transistors on it within 2 years with advancements on how we were manufacturing these chips we' get it up to eight transistors and it would be cheaper and we did that over and over again and saw massive growth in performance of our machines it was actually realistic for a bit if you took a computer that you went and bought at Best Buy and then waited 2 to 4 years bought a new one the processor could be two times faster in a very short window and it's crazy to think because now like if you buy an Apple M1 computer from 2020 and a brand new top of the line machine from 2024 the performance between those things is not that big but back in the day we saw insane improvements year over year we've started to hit walls with the physics though we realized you can only get so small for the Silicon before you start running into manufacturing problems now the manufacturing processes for a lot of these things are so complex there's only one or two companies that can even do it at the small size that we're expected to hit now if you want to make the most efficient chips possible that fit as many transistors into your die as possible you have to do that through a company like tsmc because they're one of the only places in the world that can that way and companies like Intel apple and Nvidia all rely on that one manufacturer tsmc that is still not hitting the mors law goals but they're the only ones even vaguely close we have effectively accepted that Mo's law due to physics is no longer true here we see from a study where somebody proposed a new alternative to Mor's law the blue line is Mora's law the orange line is their alternative law but the green is performance and you'll notice things are pretty hard up here when before they were going up at a steady rate from the' 70s to the 2000s even like the 2015 things were going up pretty steadily but we've started to see a flatline and the harsh reality is that from 2020 onwards it's gotten worse not better that's terrifying obviously there are companies that disagree here is a diagram from Nvidia where they actually admit that the CPU performance we're seeing has pretty hard where we went from getting multiple giant wins to around 1.5x per year down to like 1.1x per year I'm streaming right now using a PC with Hardware from when did the 10700k release date yeah the processor of my desktops from 2020 it's not even one of the high-end ones and it doesn't perform much worse than the top spec one I just bought a new computer in different room performance wins year-over-year have gotten way worse even though the technology is still advancing we're still making big wins in the manufacturing Intel and AMD are as competitive as ever we're still not seeing massive wins anymore this does have benefits though like if you buy an old processor for way cheaper you still get really good performance you can buy a MacBook Air M1 from like Walmart used for $400 and have great performance on a machine that I paid 2 Grand for not long ago and those things are great I know people are pitching all about this isn't about performance as transistor count we've used the number of transistors as a way to measure the progress of processors and historically if you had a big win in the manufacturing process if you made the dies go from 10 nanometer transistors down to like 4 nanometers you would see massive winds this is rough obviously nvidia's doing their thing here claiming that GPU compute performance continue to grow the fun thing with graphics cards is they don't have the same model with cores with the complexity of sharing things between them because the cores in a GPU are significantly Dumber it's a different abstraction which means you can just staple more and more gpus onto each other to improve performance you might end up with a GPU no longer being a tiny little thing you slide into your computer and now it's a giant room full of things it's still one GPU because of the way the chips are architected but the only way Nvidia is going to see this type of performance wins continuously is if they just add more and more chips to their actual architecture it's kind of cheating but the reality is that the tech that we use today which is traditional CPU manufacturing we have hit a physics wall for how much we can see and the only way to get out of it theoretically is an entirely different architecture and way of building compute things that rely on this model will not benefit from these advancements as much but anything that can work with this model could theoretically continue to see growth on that note gpus are not necessarily the best way to do AI stuff just a quick tie in I think it's interesting that IBM is researching analog AI chips similar to the stuff that we saw with Bitcoin back in the day where before you would mine Bitcoin with a GPU before as6 were made which were specialized computers just to make Bitcoin mining as efficient as possible we're starting to see some research into doing this for AI as well which is exciting potentially gpus aren't the right architecture for AI and we can see advancements and these chips once they work will probably Advance significantly faster than CPUs or gpus so why am I talking about all of this when I'm talking about models hell why am I even talking about models I saw a very interesting post for mistol mol is one of the two big open-source AI businesses it's them and funny enough meta So Meta faceb are working on llama which is their open- Source model it's technically not open source because you can't run the code yourself but you get the model and you can use it however you want mol is doing the same thing and they just released mol large 2 the new generation of their Flagship model compared to pror mist large 2 is significantly more capable in code gen mathematics and reason it also has stronger multi language stuff and function calling stuff cool the key here is large enough this made me start thinking a lot about the plateau that we're likely reaching and I'm not the only one thinking about this here's a tweet from Yan laon who is the head of AI and llm research at Facebook and meta he's one of the ones most directly responsible for the creation of llama and he said if you're a student interested in building the next generation of AI systems don't work on llms what llms are how all of these things work well let's rephrase this if you're a student interested in building the next generation of computers don't work on CPUs or don't work at Intel it's obvious when you look at the numbers that iteration on CPUs is not going to be where we see massive performance wins and massive computation wins going forward different architectures will have to be invented and iterated on for us to see meaningful improvements in performance year-over-year Apple does this in all sorts of interesting ways one of the crazy things Apple invented was the idea of having different cores with different roles so you had efficiency cores that are trying to use as little power as possible to do simple things and then performance cores that use way more power but are quite a bit more powerful they also started embedding things like video processing and video encoding chips that just do h264 h265 decoding and encoding way more efficiently Apple started adding things to their processors that weren't just CP and also weren't just gpus in order to optimize specific things so they could keep seeing massive performance wins I think this is the future for AI as well and I have a reason I have a very similar chart to this one notice how much smaller the winds are getting Claude saw another solid one with Sonet in 3.5 but the Gap from gp4 Turbo to Turbo 2 to 40 is a lot smaller than from four turbo to four it is way smaller than from 4 to three CLA one to two to three some massive winds but those are starting to slow down as well we're seeing a plateau of the quality of the responses these models are generating it is not like going from 4 to 4 Turbo to 40 was less work than going from 3.5 to 4 if anything there is more money more time more gpus more effort going into these bumps and the actual bump we're seeing is going down so each of these ations takes more money more time more compute more energy and the results are not as big as they used to be I know a lot of people are saying the AI future is going to Doom us all because the AI keep getting so much smarter eventually they're going to be smarter than all of us I don't see that here I don't see that here at all what I see is a theoretical ceiling that we're getting very close to and a closing of the Gap in performance between these different models more and more these options are going to become Commodities the same way you have like 15 different computer manufacturers is making the same Windows laptop that has roughly the exact same performance we're starting to see that here too I have to read a LinkedIn post which I know pain cringe miserable so I'm going to soften the blow with an XK CD first this one was linked in chat and I thought it was really funny number of computers created is going up a lot year-over-year in fact I think it's going up exponentially but the number destroyed by hurling them into Jupiter it's a much smaller number it's only three so far NASA needs to pick up the pace if they ever want to finish the job yeah they ever want to catch up they got work to do it's a fun way to think about data in these ways the compute changes over time anyways the bitter lesson famous 2019 blog post claims that General AI methods using massive compute are the most effective nvidia's soaring stock price supports the thesis but is this approach sustainable what are the in the original blog post AI Pioneer Rich suon makes the following observations over the last 70 years AI researchers have repeatedly made the same mistakes of trying to bake human knowledge in into AI systems only to be eventually outperformed by a more General method using Brute Force compute this is funny cuz we're seeing the opposite in processors now where processors were trying to just increase how many transistors were in them and how fast they could solve problems and now we're seeing specialized chips being embedded in the processors that do certain things way better some prominent examples of what was happening before with models were custom chess and go engines versus deep blue and Alpha zero this was a fun one the go not the programming language the board game was really hard for software developers to solve because the game has so many different potentials you can't just encode all of them and then figure out which is optimal and we learned after trying to make custom engines for these things that AI Solutions like deep blue and Alpha zero that were more generic more traditional AI did a better job than the custom code we wrote it took hilariously more compute to do it like hundreds of times more but the results were always better the main reasons for this are the following building an expert knowledge is personally satisfying for the experts and often useful in the short term it's a very good point if you have experts that know this game really well or Know video encoding really well they can Flex their knowledge feel useful and see an immediate result all of which feels good on top of that researchers tend to think in terms of fixed availability compute when it's actually increasing daily this is also a fair point yes the amount that a given processor improves year-over-year has gone down but the amount of processors you have available is going up especially with Nvidia going insane with their manufacturing Sun concludes that we should focus on on General AI methods that can continue to scale most notably search and learning we should stop trying to bake the contents of the human mind into AI systems as they are too complex and instead focus on finding metame methods that can capture this complexity themselves some of the important things that people pointed out are that mors law is fading architecture of our most successful learning models were actually carefully handcrafted by humans like Transformers comets lstms Etc and for General computation problems like integer factorization progress based on human understanding was often far greater than progress according to Mo's law another great point we're still optimizing algorithms in ways that we never would have imagined possible before one that I love to cite here is the fast inverse square root which was used in Doom in order to handle lighting Reflections and rendering because knowing the inverse square root lets you know how far something is relative to multiple points and it's used a ton for doing math in games previously getting this number getting the inverse square root took a lot of compute and as such the idea of 3D games was basically impossible but someone discovered a math hack they didn't even understand at the time the fast inverse square root function that was in this code base had evil floating Point bit level hacking it's the comment here this weird bit shift where they take this random hardcoded value subtract the bit shifted long long representation of Y comment what the [ __ ] next comment first iteration where we multiply it by three haves and this function here and we could run it again if we wanted to be more accurate 3D Graphics program is supposed perform millions of these calculations every second to simulate lighting when code was developed in the early 90s most floating Point processing power lagged the speed of integer processing so yeah if you were trying to do this with floating points which everyone was it would eat your processor the advantages in speed in this fast function came from treating the 32-bit floating Point word as an integer then subtracting it from a magic constant this integer subtraction in bit shift resulted in a bit pattern which when redefined as a floating Point number is a rough approximation of the inverse square root of that number this function this crazy math hack allowed us to add Dynamic lighting to 3D games this wasn't something we got because processors were way more powerful it was a clever hack that allowed us to invent a new genre of game effectively pretty nuts pretty crazy stuff that this enabled as much as it enabled because somebody came up with a clever math hack that's not even that accurate it's just accurate enough so as is said here the wins we saw on compute the revolution in 3D games that we saw after that code came out and people started using the engine that wasn't because gpus or CPUs got way better it's because our understanding of how to use them to do these specific things got better and we saw massive wins not because the CPU got way faster but because we found smarter ways to use it and I think this is going to be true now more than ever in the same way we're reaching the cap of how much you can do with a C CPU we're reaching the cap of how much you can do with an llm companies like open AI show that focusing on more compute may still lead to massive gains as compute power despite the warning of Mo's law continues to increase several orders of magnitude over the next decades don't necessarily agree currently the hype is definitely outperforming Mo's law see the image below as a result AI is at risk of creating a deep environmental footprint and research is increasingly restricted to large corporations that can afford to pay for the compute it's a bitter lesson of the last year yeah this is a fun one Mo's law versus AI popularity but again Moors law is plateauing and AI is now way more popular than what Mor's law enables so we're just spending billions on gpus found a surprisingly good chart from Gartner believe it or not the hype cycle for artificial intelligence hype Cycles are very common this particular chart the startup hype cycle an idea happens we have a spike of excitement first Valley of Death happens where you realize this is hard you go hard you go really [ __ ] hard you get inflated expectations irrational exuberance and then pain you end up in this thing called the trough of disillusionment where you're unsure of everything then the slow slope of reality as you figure out what you're actually capable of and what your product company Vision whatever it is actually could resolve to and then you hit the real company and real value so back to the Gartner chart it's funny they have all these examples in here first principles AI multi M agent systems neuros symbolic AI more and more things happening and we start getting into generative Ai and then we hit a massive Point realized we needed more optimization things like synthetic data better model optimization AI that is on the edge so to speak so it runs on our phones instead of on the servers knowledge graphs but you notice we're going down because these things aren't fun these things suck and they're necessary for us to keep evolving then we started seeing AI makers and teaching kits to try and get people to actually learn [ __ ] autonomous vehicles with which were very painful and still are cars that drive themselves are far from functioning but now we're seeing more and more things that will hopefully allow us to really benefit from AI but we need to make sure our expectations are realistically set not around the exponential growth every year rather around how we apply the functionality of these things to actually benefit Our Lives day in and day out I am honestly just annoyed that people pretend the models are going to get two times better every couple years because we went through that that's clearly over we're just not seeing levels up like that anymore what I'm expecting us to see instead is massive winds in things that we're not currently using models 4 like we're starting to see video generation catch on and it's taking us a lot of time to get there but I could see us growing there really quickly similar to how chat GPT got way better really quickly but it will also hit a plateau and I think we're going to see more and more of those plateaus hit and our solution isn't going to be magically make it better it's going to be entirely different models and hybrids where we take advantage of handwritten and crafted code maybe human massaging of things and AIS and intermingling and mixing those the same way CPUs and gpus take turns working on things depending on what each is best at handwritten code and AI code doing similar stuff has a ton of potential and I think that's going to be the future of AI because this this is not the future of AI this is a flatline this is a plateau This Is Us reaching the end not the beginning and if mistol is saying that their model's large enough I'm inclined to agree especially when you look at the numbers here and you see how close all of these models are getting to being basically even the winds are no longer the models being way better than the others the winds are going to be efficiency performance speed of responses and then the next set of wins is going to be how we use these things in new and unique ways this is actually a very interesting link there's a project called The Arc prize that was just linked from chat AGI progress has stalled new ideas are needed it's a million dooll public competition to beat an open- Source a solution to the ark AGI Benchmark most AI benchmarks measure skill but skill is not intelligence general intelligence is the ability to efficiently acquire new skills charlot's unbeaten 2019 abstraction and reasoning Corpus for artificial general intelligence is the only formal Benchmark of AGI it's easy for humans but it's hard for AI oh this is fun this is this is going to be like captions basically so we have these patterns an input and an output it's pretty clear what we do here configure your output grid there and then we have to put the dark Blues here here here and here submit fun so the point here is these are the types of puzzles that we can Intuit it we look at the pattern and we can learn quickly what the pattern is with these things it looks like the light blue is ignored red has the outward pattern then dark blue has the pattern with the like t-shape but AI is historically really bad at solving these types of things so here's the arc AGI progress but if we look at other AI benchmarks that people use a lot of the ones we were looking at earlier like H swag imag net all of these it seems like things are improving at an insane rate when you look at general intelligence through a benchmark like this AI sucks at it progress towards artificial general intelligence is stalled llms are trained on unimaginably vast amounts of data yet they remain unable to adapt to simple problems they haven't been trained on or make novel inventions no matter how basic strong Market incentives have pushed Frontier AI research to go closed Source research attention and resources are being put towards a dead end you can change that I like that they have a they call out that the consensus definition for AGI is wrong AGI is a system that can automate the majority of economically valuable work but in reality AGI is a system that can efficiently acquire new skills and solve open-ended problems yes that's what the General in AGI stands for I actually fully agree with this call out defition are important because we turn them into benchmarks to measure progress towards AI I fully agree I love that Nat fredman is one of the advisers the old CEO of GitHub we also have Mike knop who's an absolute Legend who's been involved in all things software Dev and AI for a very long time yeah I love this and I think this is the only way we're going to really see improvements and wins with AI llms are hitting their limitations and as we saw here they're not really winning on General benchmarks like this and sure we have these fancy benchmarks that everybody loves but even these we're starting to see a flatline and a plateau on them we might be at the end of the llm Revolution and if we want to see AI continue to grow and advance in its capabilities we might have to leave behind llms the same way we're starting to leave behind CPUs the future isn't an llm but faster if we want the future to be AI it has to be a different type of AI let me know what you guys think and tell me all the ways I'm wrong until next time peace nerds

💡 Tap the highlighted words to see definitions and examples

關鍵詞彙(CEFR B1)

plateaued

B1

To reach a stable level; to level off.

Example:

"plateaued pretty hard where we went from"

improvement

B2

The act of improving; advancement or growth; a bettering

Example:

"can do with them the improvements from"

speculation

B2

The process of thinking or meditating on a subject.

Example:

"speculation from a Dev and a cardware"

enthusiast

B1

A person filled with or guided by enthusiasm.

Example:

"Enthusiast back in the like what' 7s and"

observation

B2

The act of observing, and the fact of being observed (see observance)

Example:

"observation is that the number of"

difference

B2

The quality of being different.

Example:

"difference between those things is not"

manufacture

B2

The action or process of making goods systematically or on a large scale.

Example:

"manufacture that way and companies like"

alternatives

B2

A situation which allows a mutually exclusive choice between two or more possibilities; a choice between two or more possibilities.

Example:

"Alternatives in the original blog post"

flatlining

B1

(of the heart) To stop beating.

Example:

"flatlining pretty hard up here when"

relatively

B1

Proportionally, in relation to some larger scale thing.

Example:

"relatively steady rate from the' 70s to"

想要更多 YouTube 聽寫練習?造訪 練習中心.

想要同時翻譯多種語言?造訪Want to translate multiple languages at once? Visit our 多語言翻譯器.

聽寫文法與發音技巧

1

Chunking

注意說話人在特定片語後的停頓,有助理解。

2

Linking

聆聽連音,當單字連在一起時。

3

Intonation

留意語調變化以掌握重點資訊。

影片難度分析與數據

分類
science-&-technology
CEFR 等級
B1
時長
1330
總字數
4321
總句數
625
平均句長
7 詞

可下載聽寫素材

Download Study Materials

Download these resources to practice offline. The transcript helps with reading comprehension, SRT subtitles work with video players, and the vocabulary list is perfect for flashcard apps.

Ready to practice?

Start your dictation practice now with this video and improve your English listening skills.