davepeck.org

Thu, 04 Apr 2024 18:23:00 GMT

Just ran across a nuanced older Cory Doctorow article asking what kind of bubble AI is:

Of course AI is a bubble. It has all the hallmarks of a classic tech bubble. Pick up a rental car at SFO and drive in either direction on the 101 – north to San Francisco, south to Palo Alto – and every single billboard is advertising some kind of AI company. Every business plan has the word “AI” in it, even if the business itself has no AI in it.

...

Tech bubbles come in two varieties: The ones that leave something behind, and the ones that leave nothing behind.

Doctorow argues — and I agree — that unlike, say, the crypto bubble, AI is likely to leave value behind when it pops.

He also makes an interesting argument that, because there are so many low-stakes low-dollar uses for AI, smaller self-hosted models may be the long-term winners:

There will be little models – Hugging Face, Llama, etc – that run on commodity hardware. The people who are learning to “prompt engineer” these “toy models” have gotten far more out of them than even their makers imagined possible. They will continue to eke out new marginal gains from these little models, possibly enough to satisfy most of those low-stakes, low-dollar applications.

I disagree with the article on a couple points, however. First, Doctorow worries about the origins of small models:

These little models were spun out of big models, and without stupid bubble money and/or a viable business case, those big models won’t survive the bubble and be available to make more capable little models.

The past many months have seen advancements in low-cost training that make me think this probably won't be an issue.

Second:

The universe of low-stakes, high-dollar applications for AI is so small that I can’t think of anything that belongs in it.

Triple-A game titles and other high-end entertainment strike me as straightforward examples. And, with Microsoft integrating large models across their entire Office suite, perhaps another answer is "all those low-stakes Excel spreadsheets and Word docs that nevertheless do something useful for business somewhere".

There's a lot more to Doctorow's article; it's worth a read in full.

Count More

Fri, 23 Feb 2024 00:18:00 GMT

If you're a student in the United States, you can register to vote either in your home state or your school state.

If one of those happens to be a battleground state in the 2024 presidential election, your vote effectively "counts more" there.

That's why we built Count More, a tiny website to help students choose:

It's maybe the (technically) simplest project we've worked on at Front Seat but, with the right campaign behind it, we hope it will meaningfully impact the 2024 presidential election.

Fri, 17 Nov 2023 19:55:00 GMT

Angie Wang's sketch in the New Yorker, "Is My Toddler a Stochastic Parrot?", is a lovely meditation on language, AI, and life:

The world is racing to develop ever more sophisticated large language models while a small language model unfurls itself in my home.

Seriously, just find a large screen and read it. It's such a delight.

OpenAI: Consumer & Cloud

Thu, 09 Nov 2023 17:20:00 GMT

Sam Altman, partway into his understated keynote at the first ever OpenAI DevDay:

Even though this is a developer conference, we can't resist making some improvements to ChatGPT.

Before this moment, I only fuzzily understood that OpenAI operates in two separate but related markets. They have:

A consumer product called ChatGPT. This includes mobile apps, websites, secondary models like DALL·E 3, and integrated tools like the web browser and code interpreter. ChatGPT, used by 100M weekly actives, costs $20/month. Despite Monday being "DevDay", OpenAI launched major new consumer-facing features. Most notably, it now allows anyone to build custom "GPTs".
A growing suite of cloud services. Before OpenAI DevDay, I might have referred to this as the "OpenAI API" — a small piece of code that interfaces with OpenAI's foundation models. Now, this seems too simplistic as the API quickly evolves into multiple value-add (and margin-add!) services like the Assistant API. This is similar to how AWS adds value (and margin) on top of its few core services. OpenAI's cloud, with 2M registered developers, undoubtedly generates significant revenue.

Finding oneself in two very different product and market segments — suddenly and at scale — is no small feat! Yes, these markets utilize the same underlying technologies. However, their pricing models, sales strategies, and value propositions are quite different and are likely to diverge over time. This isn't exactly unheard of in the tech industry, but it's a significant undertaking for a company that's been around for less than a decade.

Sam Altman, with charming understatement:

About a year ago [...] we shipped ChatGPT as a "low-key research preview"... and that went pretty well.

Very few consumer services reach ChatGPT's scale. In under a year, OpenAI accidentally defined a new consumer product category and became its heavyweight. I don't think we need to look any further to find the future of consumer chatbots. Especially with the introduction of custom GPTs, it’s hard to see much room for creating differentiated consumer-facing chat services. Instead, developers will need to bring their unique data assets and external compute capabilites to the ChatGPT interface. The Canva demo is a compelling example: Canva is a decacorn but the expectation is that users will still start in ChatGPT.

As for the growth of a new kind of cloud, focused not on virtual compute and object storage but instead on foundation models, we should expect to see many more value-add services in the future. The logic behind this year’s launches seems straightforward: OpenAI simply observed the emerging architectural patterns behind LLM applications and implemented versions that are better and easier to use, readily available from the same vendor that provides the LLM itself. Like AWS, OpenAI is the heavyweight in this emerging cloud segment. While there will always be room for smaller players along competitive axes like model customization and privacy, and I hope we see plenty of innovation there, the default place to start building services that need LLMs will probably be OpenAI for the foreseeable future.

Wed, 01 Nov 2023 23:40:00 GMT

Mike Hoye, writing on Mastodon:

People go to Stack Overflow because the docs and error messages are garbage. TLDR exists because the docs and error messages are garbage. People ask ChatGPT for help because the docs and error messages are garbage. We are going to lose a generation of competence and turn programming into call-and-response glyph-engine supplicancy because we let a personality cult that formed around the PDP-11 in the 1970s convince us that it was pure and good that docs and error messages are garbage.

Mike takes a look at what can go wrong when writing a one-line "Hello World" program in C. It's a darkly comic example of the slapstick violence that developers inflict on one other.

It's not just error messages and documentation. Today's tools and frameworks overflow with violence. Its omnipresence inures us to it; we cast blame anywhere but where we should. All developers suffer for it but new developers suffer disproportionately more.

Mon, 23 Oct 2023 17:09:00 GMT

Anil Dash gave a delightful keynote address at last week's Oh, the Humanity! conference.

I checked and was not surprised to learn that Anil and I are very close in age. We were kids when personal computers were new and we were probably both in middle or high school when the web was born. We seem to have similar perspectives on the why of the open web: why building a more open web is — in addition to being fun — an important public good.

Anil's framing is very personal, though, and I found it very moving. The full talk is available on YouTube and feels like it deserves more than its current ~650 views.

Otis Health

Tue, 28 Feb 2023 18:56:00 GMT

Today we launched our newest PSL spinout, Otis Health.

In its first iteration, Otis offers a free discount pharmacy card to the underserved 1099 worker market. A shocking percentage of contract workers have no insurance whatsoever; saving even a handful of dollars on medications can make a meaningful difference. Otis can often save much more than that.

I'm excited about Otis on three fronts. First, it brings a modern design and user experience to a market that until today has sorely lacked it. Second, Otis' discount card is free to use; all the economic magic happens behind the scenes, in the form of complex contracts between retail chains, benefits managers, distributors, and manufacturers. Finally, Otis has its eyes firmly fixed on several adjacent services that we think will also meaningfully improve the lives of 1099 workers in the future.

Oh, and one more (the most important!) reason to be excited: the team. It's been a blast working with Aaron, Luke, Sanford, Sharon, and Steve to get this thing out the door.

Thu, 26 Jan 2023 18:11:00 GMT

I set up a personal Mastodon instance.

I'm now @davepeck@davepeck.org in the Fediverse.

I've been watching Mastodon since early 2018 but only recently took the plunge. Mostly, that's because I've gone in circles contemplating how this blog — and its built-in micro sub-blog — should tie into the broader ActivityPub and IndieWeb universes. I'm still unsettled on that score but figured it was well past time to do something.

So: here we are. Hello 👋 Mastodon.

Sun, 15 Jan 2023 23:40:00 GMT

Craig Hockenberry, writing on his long-lived personal blog:

Well, it happened.

We knew it was coming.

A prick pulled the plug.

Over the weekend, Tweetbot, Twitterrific, and every other popular third-party Twitter client was unceremoniously banned. It's a stupid petty move on Twitter's part, executed in an impressively stupid petty way. I imagine it's the final nail in the coffin for several high-profile Twitter hangers-on.

Most of the people I follow, though? They're long gone.

Fri, 06 Jan 2023 23:03:00 GMT

Jesse and Anthony wrote a new post on PSL's blog describing five AI-powered tools that we've been making increasing use of in the studio. It's amazing how fast this space is moving.

Fri, 23 Dec 2022 21:55:00 GMT

Ran across a software blog post that made me feel, well, old:

In traditional web applications, web pages are rendered on the client side. The browser receives a blob of JavaScript from the server, processes it, and paints the UI that the user sees.

This is not the "tradition" I grew up with!

(It's also not a "tradition" I've ever particularly loved.)

Mon, 12 Dec 2022 22:56:00 GMT

Me, five weeks ago:

For me, it's not really about Elon.

Musk, yesterday:

My pronouns are Prosecute/Fauci

Yeah, well, I take it back. It is about Elon. Once upon a time, Twitter was fun. It's a shame a midwit troll had to come along and wreck the party.

I suppose all things must end. Onward.

Thu, 08 Dec 2022 16:15:00 GMT

This is an audio post. You can listen to the audio here: Cats Rule (Dogs Drool)

Definitely the best and most important bit of music my 8yo and I have made together:

Phase Transitions

Thu, 01 Dec 2022 16:17:00 GMT

GPT-3 shipped two years ago. Its capabilities, and that of descendant language models like OpenAI Codex, astonished me on day one; two years later, I'm still just as astonished.

Earlier this week OpenAI announced ChatGPT, a new variant intended to be used in interactive dialogue. With ChatGPT I find myself astonished all over again.

One delightful discovery is that back-and-forth conversation is a good way to build code. It's a form of literate programming that Knuth probably never imagined.

I paid for GitHub Copilot the moment I could. Copilot, which builds on top of Codex, is more than just astonishing: it's useful. Yet, as impressive as Copilot is, playing with ChatGPT makes it clear that Copilot has the potential to do vastly more even today.

The Internet has done its work and there are too many fun ChatGPT code samples to choose from. The most compelling examples refine code just by talking it over. There are also countless short examples that might have been accomplished with the previous generation of Codex or GPT-3 but that seem to have renewed potency today, like explaining a bug or exploiting a buffer overflow. But if I were to highlight just one example that seems to capture the moment, it might be this complete absurdity from Riley Goodside:

Computers pretending to be gangsters with a knack for complexity theory. That is where we're at today.

We're at one of those handful of moments when our industry undergoes a deep and lasting phase transition. It's easy to draw parallels with previous transitions like the advent of the Web in 1991. Then, as now, a new technology was introduced that collected recent advances into a package that felt wholly new. Even in its first version, that new technology was instantly useful in spite of its obvious flaws. It was easy to imagine a long road of improvements ahead. Most of all: people couldn't look away. The Web struck like lightning. Large language models? Much the same.

Having said this, there's also something that feels entirely different to me about this moment. We're playing with language. Language is primal. It is quintessentially human. It is fire. It's no accident that GPT-3 was built by an organization whose stated ambition is to develop the world's first artificial general intelligence.

What lies on the other side of this transition? I don't know; until we get there, I'll have to content myself with those fast-talkin' wise guys. Hopefully they can teach me another thing or two about computer programming along the way.

Wed, 30 Nov 2022 05:36:00 GMT

This is an audio post. You can listen to the audio here: Gymnopedie No. GPT-3

It's snowing outside. I decide to listen to A Charlie Brown Christmas. A bit early, but still: snow!

At the same time, I'm playing with GPT-3's new text-davinci-003 model. It's impressive.

A couple weeks ago, Adam and I tried to get GPT-3 to spit out chord progressions. Today's updated GPT-3 seems pretty good: sure, some of my prompts result in sonic (or textual) mayhem, but many more result in ii-V-I and other legible changes.

I punch in an oddball prompt:

Give me the chords to a straight-ahead jazz reharmonization of Erik Satie's Gymnopedie No. 1. A simple list will do.

GPT-3 replies:

Ebmaj7 - Bbmaj7 - Ebmaj7 - Bbmaj7 - Cm7 - Fm7 - Gm7 - Bb7 - Eb7 - F7

Okay, those first repeating chords make some sense; I can tack a melody to that. The next are a bit mysterious, but I give it a go and eventually find a way to make them fit. I toy with chromatic approaches while Vince Guaraldi sinks in and somehow end up with... well, "Gymnopedie No. GPT-3" I suppose.

Thu, 03 Nov 2022 16:14:00 GMT

I'm taking a break from Twitter.

My usage tailed off a while ago when I realized that my old-school RSS reader was once again both more valuable and more joyful. Blogs have had quite the resurgence in the past three or four years; I want to be a part of that!

For me, it's not really about Elon. The timing just feels right.

Mon, 30 May 2022 16:29:00 GMT

Yesterday was very rainy here in Seattle so the 7yo and I made "The Cat Food Game". It's Sokoban but, y'know, with cats. If anyone's curious, the code is here.

Wed, 18 May 2022 17:01:00 GMT

My 7yo spends a lot of time making fun programs in Tynker. Most of them involve animated cats. She's a true self-directed learner but every once in a while I get to share a new idea. This weekend it was recursion. We decided to make it "visible" by implementing a simple tree drawing toy. (Later, I went back and made a "fancy" variant.)

Wed, 27 Oct 2021 17:19:00 GMT

In case it's not clear from my previous post, I'm an unrepentant crypto bear. At the same time, I'm fascinated by crypto's culture and by its technical edifices: great walls of complex code all in service of... well, I'm not sure what, exactly.

One thing I haven't done is backstop my bearishness with a financial position. Thankfully, I can live vicariously! Paul Butler's rationale for and approach to shorting bitcoin by shorting publicly traded miners is a breath of fresh air.

Down the Crypto Rabbit Hole

Mon, 18 Oct 2021 18:05:00 GMT

Crypto is a wild mix of culture, code, and capitalism. Its proponents speak of "going down the rabbit hole" in a positive way, as if a fundamental truth awaits on the other side. Crypto's rabbit hole is deep, yes, but in my view it is full not of pleasant truths, but of distressing surprises that seem to animate the surrounding chaos.

On its surface, crypto's "culture" feels both youthful and vibrant. I'm reminded of the early days of the web, when sites were weird and creators built without constraint. Then as now, nobody quite knew which way the industry's winds blew; they built anyway. Crypto is the center of a new design trend that hearkens back to Space Jam, bringing its aesthetics into a technotronic future. It's home to new writing and criticism that feels like the work of whip-smart semiotics majors. Like the early web, there's lofty rhetoric about crypto's utopian potential. New social clubs seem as enthusiastic about throwing underground raves in Paris as they are with the market value of their tokens.

This bizarre emerging culture arrives with what seems to me to be a giant asterisk. I think @pinboard said it best when he described crypto as "an unregulated casino with a hip bar scene". He continued: "there's nothing morally neutral about the criminality at the heart of the endeavor". Every big-time speculator, punter, grifter, money launderer, and ransomware bandit in the world is hard at work in the casino. No matter how literate the bar scene, it's the fruit of a poison tree.

Just inside the crypto rabbit hole, we find a genuinely new and expressive culture emerging from the grimy bits of unregulated capitalism run amok.

Code animates the crypto ecosystem; its more starry-eyed believers claim that code is all that matters. And, to be sure, there is very interesting code behind crypto. A dozen years ago, Bitcoin introduced the world to a new form of distributed consensus. Half a decade later, Ethereum proved the viability of smart contract programming. Since then, algorithmic advancements have led to low-cost high-throughput chains. Emerging financial protocols on top of blockchains — MakerDAO, which creates a dollar-stable asset through collateralization; Uniswap, an exchange without an order book; and Compound, a decentralized interest rate market — all have innovative technical and economic underpinnings. (Are they long-term valuable underpinnings? I admit my skepticism.)

The technical history of the Internet is, in part, the history of the birth, adoption, and stewardship of distributed protocols by its broader community. Blockchains follow in this tradition but also depart radically from it: they are the first protocols to arrive with an asset class attached. Protocols like SMTP and HTTP created immense value for the world but captured little for their inventors; blockchains and the protocols built on top of them upset this balance, allowing inventors to capture considerable value for themselves. Historically successful protocols typically see slow early adoption; crypto's new protocols break this mold by nearly requiring substantial up-front speculation (or wise investment!) in order to achieve escape velocity. As a result, it's alarmingly easy to launch systems that look indistinguishable from Ponzi schemes or multi-level marketing.

Midway down the crypto rabbit hole, we find a technically intriguing class of Internet protocols that upend the historical balance of value creation and capture, leading to surprising and problematic new dynamics.

Ultimately, healthy economies need productive ends. While speculation almost certainly drives crypto's outsize market cap today, it's hard to ignore the community's rapid experimentation. Perhaps crypto will never uncover true sources of value creation; perhaps there are none to be found. (That's certainly my instinct at the moment!) Or perhaps the essential ingredients are already in the kitchen. At the heart of every crypto experiment lies the simple abstraction of tokens: marks on ledgers maintained by the network. Anyone can wave a "magic wand" and declare the existence of a new token that the world will honor merely by collective agreement. Of course, waving wands alone isn't enough to create value. Yesterday, we experimented by attaching goofy images of punks and apes to our tokens. Today, we're starting to staple metadata with complex structure. Tomorrow, perhaps, we'll wrap our tokens with increasingly sophisticated code: code that grants permissions and access rights, code that defines and enforces behavior, and code that describes how to interact with crypto's new payment rails. There's an immense design space to explore; it may be too bearish to entirely dismiss its potential.

The collective agreement that makes crypto's "magic wand" possible at all is, to me, a puzzle. By nature, digital content can be reproduced at zero marginal cost. Why is it that we've decided to recreate scarcity in the digital world? Yes, we've grappled with digital scarcity in limited domains before. Video games sell digital items; the appeal is easy to understand. Software phones home to check its license. Intellectual property itself is a form of manufactured digital scarcity. Yet crypto seems to throw these doors wide open. Perhaps this was inevitable. Perhaps the centuries we've spent building societies and economic systems around the unavoidable problem of physical scarcity makes digital scarcity feel natural to us even though it's unnatural to information itself. Whatever the case, the deluge of capital that floods the crypto markets today practically guarantees that we'll experiment with manufactured digital scarcity for quite some time to come.

Deep down the crypto rabbit hole, we see that while crypto might portend a revolution, at its heart is a simple reassertion of scarcity in the digital realm — an assertion that is by no means a foregone conclusion.

Where does crypto go from here? With so many cards — cultural, technical, economic, and regulatory — in the air, it's impossible to predict its future trajectory. But the technology industry is immensely path dependent. Particularly when buckets of money appear, feedback loops can form whose outcomes seem all but inevitable. Speculators and venture capitalists alike have inundated crypto with cash. Blockchains may become the future only because their story was told, speculated on, invested in, and told some more. Or they may disappear quickly, in the bursting of speculative and fraudulent excess. Whatever drives us, if society decides to wander further down this road, we should expect to see the same unequal outcomes we see today merely reproduced in the digital realm. Despite the utopian rhetoric, economic power always concentrates: crypto may mint new winners, but the tune will stay the same.