Symposium update, Kimball modeling, quality koalas, etc.
Symposium update, Kimball modeling, quality koalas, etc.
Et Cetera #1
This is a bonus dose of Data People Etc. miscellany for the month of March. Some outside posts, Data Council recs, and the debut of the Quality Koala. Enjoy!
New Whatnot eng blog — Same Data, Sturdier Frame: Layering in Dimensional Data Modeling at Whatnot — coauthored with the incredible Alice Leach and Lalita Yang. The post addresses how and why we rolled out an intermediate layer of data models following Kimball design principles.
I’ll share some context here: I was against this project at the start. I had drank the “centralized warehouses are bottlenecks” kool-aid. I don’t understand surrogate keys, I can’t remember the difference between type 1 and 2 anything, and I’m suspicious of data modeling hoo-ha.
Just look at the “basic” techniques on the Kimball site.
Junk, outriggers, and degenerates — that’s what pops out at me. Not the riff-raff I want to associate with.
But, obviously, these things are important — just not the first important things. By deferring the thought until a year in, after we had multiple teams building critical products on top of the application data, it was clear which 10% of the data was providing 90% of the value, and we could focus on modeling that portion in an “as simple as possible, but no simpler” manner, while still providing clear ownership, consistent standards, and scalable performance.
Speaking of Whatnot, check out our team’s speakers if you’re heading to Data Council this year. We are sending our best and brightest (i.e. not me).
Extreme Self-Service: Turning Data Consumers into Data Constructors, by Dr. Alice Leach
Data Contracts in the Modern Data Stack, by Zack Klein
If you see Zack or Alice around, give them a high five and ask what they’re selling on their next Whatnot live.
Is the orchestrator dead or alive?
We’ll have our first takes next week, when the Symposium kicks off, and woof, it’s a great lineup. For the next 5 weeks or so, you’ll see posts on both Monday and Wednesday.
Some symposium etiquette:
This is a drinking party, so think ahead! If you normally read Substack in the office on Monday morning, make sure to hide a mimosa in a drawer.
If you like it, show it! Show guest authors some love by mashing that like button.
If you dislike it, show it! Give guest authors a (respectful) earful in the comments.
The hypothesis we’re testing is that it’s more fun to write as part of a community, and as part of an ongoing discussion, than solo. That means you, the reader, are as much a part of it as the authors. Cheers!
I have been reading through Venkatesh Rao’s Art of Gig books recently, and an essay on adapting to your client’s needs stuck out to me. In particular, this extension of Amazon’s principle struck me as worth sharing.
[Working with clients] in areas of shared maturity is even more fun. The biggest such are is being right. I got the idea from a rare non-vacuous documents of its kind, the Amazon leadership principles, which includes this one: Good leaders are right a lot. You can acutally make a triad of such statements.
Good leaders are right a lot.
Good managers win a lot.
Good employees accomplish a lot.
It is crucial to recognize the leadership often means being right but not winning. … Even with perfect play and no mistakes, they might lose.
I recommend the Art of Gig to data people — even though we don’t operate as independent consultants, we do often find ourselves as pseudo-independent mercenaries, operating outside the core business operations. The books are full of interesting takes on how you can thrive in such an environment.
Another @vgr initiative launched last week: The Summer of Protocols.
I can’t do the actual research, and I don’t know anything about crypto, which (I think) is the primary catalyst for discussion, but I love the premise that simple decision frameworks can be surprisingly influential across space and time.
My first thoughts went to metrics — are metrics a sort of protocol?
There are similarities: they are simple, accessible, and facilitate decentralized coordination. (I don’t care what you do, really, if it grows the business) Outside the company, metrics are used to make decisions about the company, with barely a reference to its internals.
Then, I wondered more generally, is “growth” a protocol?
Anyway, Summer of Protocols seems like an interesting conversation to follow along with, and I might engage these topics a bit more in the coming months.
Finally — let me introduce the Quality Koala, an advice animal you can use whenever you get that queasy “We need to do something about data quality, but we don’t know what, so let’s just do something” feeling.
It’s also great when you only know enough about a PR to give feedback on column naming, which basically summarizes my entire professional life.
The Quality Koala is brought to you by my six-year-old son. He will, with any luck, be drawing for me full-time soon.