Mine is just running a model on my laptop. It’s just amazing! I can ask it pretty much any question and it replies relatively FAST!
Before, we lacked advancements in technology because we were limited by hardware. This advancement is the opposite: our software and the math/algorithms have brought us this.
It’s possible to make it offline but one would still need a phone for the object recognition software…that software would have to be what stays offline.
Definitely doable.
I’m a little more hopeful that the future will allow for local (network free) frontier AI technology.
Being that I’m a tech enthusiast and computer science nerd I tend to live less on the bleeding edge of technology because of privacy infringing hardware. Take for example meta glasses. So many people have adopted them because they don’t care about privacy as much as I do. So they get to live with the latest and greatest.
Though, running a local LLM on my laptop (that is state of the art) has made me a little more hopeful that the future is around the corner. Who would have thought that one day we could run advanced AI on a laptop that’s able to do RAG and CAG.
I fully agree that the only hope is offline/open source systems that we can verify are working for us and not anyone else. The more complex the hardware is the more difficult it'll be to keep them safe. To avoid bugging my home it's easy enough to open up my PS5 controllers to pull the two microphones out, but I imagine it'll be a lot more work to make sure there are no radios connected to a SoC tucked away somewhere in a household robot.
I'm not sure I'd call meta glasses the "latest and greatest". Even if there were no privacy concerns I wouldn't feel left out when it comes to giving facebook the ability to plaster ads on every surface in your field of vision. The tech has a lot of potential, but the product people are using today is trash I feel better off without.
I’m not sure if this comment is AI generated but I’d like to make a point either way.
Human biology is orders of magnitude more efficient than AI; that’s in regards to intelligence and processing the world around us.
It is not only more efficient but more complex but yet simple. Therefore, the complexity of our biology is a result of efficiency. An efficiency that allows us to process the world and achieve homeostasis in the most simple way. I’d like to see a machine achieve the same without having any type of vulnerability or weakness to corruption.
I bought 192GB of DDR3 a year ago for literally $60 ($5 a stick). It's about $22 a stick now, so more like $350 today. What on earth is _anybody_ doing with DDR3?
Demand for DDR3 is up because people who want DDR5 or DDR4 but can't afford either any more are choosing DDR3 and old DDR3-compatible systems to put it in, instead of what they really want.
All memory products use many shared resources in the supply chain, so if there is high demand in one product line, others have to raise prices to compete for the resources or stop making those lines altogether.
That is to say at least you were able to buy them at $350 today, with the current trajectory there will be no supply at all in few months.
You could set up swap space on Intel Optane media, it'll be about the same performance as DDR3 and sells for ~$1/GB on the secondary market. Though it will be a lot more power hungry than Flash, let alone DRAM - so not suitable for all uses.
Optane is available in NVMe form factor that will work basically everywhere. There's also Optane persistent DIMMs that only work in highly specific systems.
Just decided to buy 8 drives for my NAS and was surprised to see nothing in stock anywhere + prices are 3-4x higher than half a year ago. Just wasted 2k eur for 8x8tb, it should be plenty enough for my NAS but I feel stupid having to waste so much money.
My main computer has 64GB. I bought that one in late 2022 or so.
Looking at the current prices, even of the same RAM, is just
insane. Those companies really need to pay us compensation
damage here. The whole "free market" notion does not work
when you have de-facto monopolies and mega-corporations abuse
average Joe and average Jane.
I forgot to add, I paid ~500 each, Samsung for the same drive is quoting $2k on their site (maybe a new sku). These were bought 2ish years ago.
Strange things are a foot at the Circle-K.
Makes prior assumptions that getting tens of gigs of ram is cheap thrown out the window. Would likely lead to super fast SSDs such as optain being way more valuable
It is one of the thing with consumer when they remember they brought it at the absolutely lowest price point when DRAM maker were bleeding money.
Those are not normal pricing. Before the pricing collapse in early 2020, 96GB DDR5 would have cost about $450 to $500. And I will need to restate again the cost of DRAM hasn't really changed much in the past 20 years. Its price just goes up and down in cycles.
So in reality it is more like going from $500 to $1300. But consumer felt it was more like going from $200 to $1300.
Crucial are already selling DRAM made by CXMT. And China are already throwing money at it. I doubt the memory bubble will burst in next 12-24 months. As in going back to money losing DRAM pricing. As they will all pivot to HBM or other money making products. But the bulk of lower end consumer DDR5 or LPDDR5 will goes to Chinese Foundry. Assuming they have figure out how to do them well. Which they have improved but are still so far away from industry leaders.
Normally memory maker will push the next DDR standard to market just to push out Chinese competitors, I am not sure it will work the same this time around. DDR5 have plenty of other usage / demands.
Historically the price has always trended downward. When I first got into computing $200 could buy you 128 MB (yes M) of ram. Really nice systems had 512 MB.
That's obviously changed over the decades as process shrinks have lead to higher memory density. We should generally expect that ram will cheaper up and until the point where process shrinks stop happening. They've definitely slowed, but they haven't stopped.
>They've definitely slowed, but they haven't stopped.
Yes if you span into 40 years. But the spot price for DRAM floor was ~$2/GB in 2008 and touched that 2-3 times over the next 15 year. It wasn't until early 2020s it broke that into $1.
Process shrinks happen but majority of DRAM part can't be shrinked by process any more.
Exactly. My first computer had 48k, yes K of ram :-). My first PC has 2MB and made all my friends jealous as they had 1MB. Amiga 500 at the time had half.
I am keeping a piece of paper that came with my Tex Murphy game which stated that one could get 32MB of RAM for as little as $700 (1990s dollars) which would drastically improve the game!
I just started messing with local LLMs and honestly I’m pretty impressed. I have a workstation laptop with an NVIDIA A1000 (6GB VRAM) and 96GB of RAM. I rarely used my gpu. Occasional CAD design or Machine Learning with OpenCV.
I ran llama3:latest and it ran pretty fast! I’m curious to see how Qwen would run on my system.
reply