> Axiom 2: You’re optimizing over a business, not over a codebase.
Well said. Ultimately I think this is where much of the communication breakdown occurs when discussing Framework A vs Framework B online.
If you’re optimizing for _code_, sure. Stress about the ms it takes to load this or that thing. Client vs server. All of it is valid discussion from an engineering standpoint. Go nuts!
If you’re optimizing for _business_, then choose something that gets the job done. Fewer moving parts - and fewer resources needed to maintain it - the better.
This is wild. I've spent the last three weeks working on this stuff for two separate clients.
Important note if you're taking advice: cache-from and cache-to both accept multiple values. Cache to just ouputs the cache data to all the ones specified. cache-from looks for cache hits in the sources in-order. You can do some clever stuff to maximize cache hits with the least amount of downloading using the right combination.
If you ever happen to study computer science, you may come across a subject called "coding theory". It introduces you to compression as well as many other topics such as error correction (Reed-Solomon, used in RAID5, RAID6 and ECC-RAM), line coding (sometimes you need to flip bits to keep the clock on both sides in sync, no clock-signal for a long time may cause clock loss) and a lot of wonderful but weird stuff.
Let us go into detail on compression:
There is a representation. US-ASCII uses 8 bits per latin letter, UTF-32 uses 4 bits per latin letter. It is just a temporal representation to the machine -- usually in memory only, it does have the same amount of information, you can save it more efficiently to disk. You would not want to save either format to disk, it is a waste of space.
Information content (I hope my translation is correct, scan Wikipedia for details) cannot be compressed. But it can be calculated. The more seldom a letter, the more information its occurence carries. As soon as each letter is not equally frequent (compare space and "q") the information density drops.
Calculation is quite simple: Count the occurence of each letter, count the caracters used (if there is no "q" in the text, you got to save one letter and its encoding) and apply some math
For some easy examples, think of morse code and Huffman coding -- not every letter needs to be encoded using the same amount of bits.
> How much data can lowercase save? #
Nothing. Either there is (almost) no information to it in the first place, in that case compression will take care of it for you. There could only be information to it if uppercase letters were equally likely as lowercase letters
> How much data can lowercase save? #
Why do you even stop at letters? You could build a dictionary of words and compress references to it. The compression efficiancy would only depend on the amount of words, regardless of case and regardless of character set. That is why entropy depends on "symbols" instead of "letters"
So the biggest problem is to actually render your cv. For this I use https://ohmycv.app. Its a nice (free) editor, and you can write CSS for the styling, works great. I source control the markdown and CSS, i have multiple versions of the same CV, so I can change content while keeping the format the same.
I really appreciate what this project is going for. SQLite with practical replication gets to be pretty dangerous for other vendors in the space. It's the last thing we really needed for our end game customers.
Unfortunately, our org is being pushed away from SQLite because our domain is highly regulated. We have a really hard time getting something like this through technical review, especially for our up-market clients. The only thing I could defend is if this was somehow part of the first party solution and code base.
I can sell SQLite to a bank if they are tolerant to single node failure semantics (I.e. vm snapshot restore). That part is no problem. Already did it half a dozen times myself. It gets tricky if they want more aggressive RPO. I cannot sell log-replicated SQLite if it relies on another, lesser-known 3rd party for that capability. I can get away with murder on my front end vendors but data vendors never seem to escape scrutiny.
I want to hate SQL Server, but it's starting to grow on me. The knowledge that I won't have to argue with a hundred+ CTO/CIOs is starting to win out over the principled nerd things.
Replicating SQLite feels like a cursed thing for many contexts - If someone isn't happy with periodic snapshots, then the nature of their business is probably pretty serious and they probably want deeper (perhaps legal) guarantees about how things would go.
If anyone wants to finetune their own Mistral 7b model 2.2x faster and use 62% less memory - give our open source package Unsloth a try! https://github.com/unslothai/unsloth thanks! :)
It seems like the author agrees with you and picked a confusing title for the article. The article ends with a set of equations:
user > ops > dev
biz > ops > dev
biz ≹ user
The conclusion seems to be that code exists in service to the end-user and the business. The last equation (≹) is a neat way of describing that both end-user and the business are equally important to the existence of the code, even though their needs aren’t the same.
I love OpenTelemetry and we want to trace almost every span happening. We’d be bankrupt if we went any vendor. We wired opentelemetry with Java magic, 0 effort and pointed to a self hosted Clickhouseand store 700m+ span per day with a 100$ EC2.
Professional managers, particularly American MBA types is what destroys businesses. Not KPIs, Agile, Scrum or whatever the fad of the moment to blame may be.
Management is a service task, it does not produce value itself. Good management is important to prioritize, allocate resources and resolve conflicts. But that is not how MBAs seem themselves, they see themselves as the most important thing around, they're the ones in charge after all. And damn physics and reality, the spreadsheets say you can get get a baby out in a month if you use 9 women to do it. Soon management captures all the attention and sucks away all the money, and shortly thereafter end up with a moribund shell of the former company. You can tell how close to moribund a company is by looking at the manager to worker ratio. Get out of there once it climbs much above 1 to 10.
There are good managers around, but they're almost as rare as unicorns.
My brother passed away very suddenly a few years ago, and I was put in charge of wrapping up and archiving his "digital" life. We were very lucky that we had access to a recovery email for his main gmail account (as well as a couple of passwords that his partner knew) and was able to access and archive virtually all data we could think of (services like Google Takeout were invaluable). I realized that if this had happened to me, it would have been virtually impossible to do, as all my passwords and credentials are in my password manager, and the password to that was only in my head.
It's a good thing to plan for this eventuality, to make it easy for your family and friends to wind up your "digital life" after you've passed. 1Password has a very good solution for this, with a "recovery document" you can print out and write down your password on, which contains instructions anyone else would need to access your 1Password account. I gave a copy of this document printed out to a small number of people I trust implicitly.
You never know when something sudden can happen to you. For the sake of those you leave behind, it's a nice gift to plan for this eventuality, even if it seems far off at the moment.
This is not a great interview question. It’s not even a good interview question. It barely evaluates any technical abilities and it ignores most technical abilities that matter.
This question could be improved as follows:
* present the naive solution. Explain that this is a PR that was submitted in 20 minutes by our new intern from Waterloo. Ask them to help improve it.
* suggest to them the intern’s argument why the lookup is O(1) and explain that it’s actually O(n). Ask them to write a PR comment to break the news to the intern.
* explain the optimal solution with tries. Ask what happens if the vocabulary is big enough—and the prod environment is resource constrained—so that the trie is rarely in memory when the function is called. How would the trie solution compare to a binary search through a length normalized list of words?
This kind of question is barely useful to interview new hires—when you still may have basic questions about how much undergrad material “stuck”. Within a few years when they hit full-performance level you should care much less about this kind of question and much more about “higher order” thinking about engineering.
How do you communicate bad news? How do you identify, communicate, and mitigate risk? How do you plan large tasks at at appropriate level without over engineering the solution? What questions should you ask at different project stages to capture the right requirements, anticipate issues, and successfully execute over time?
But whatever, I’m sure you get lots of signal out of repeating “a hash table lookup is O(1)” “nuh uh” on loop for 10 years. :eyeroll:
I did enjoy reading the preview book for EventStorming, seems like a decent format for getting stakeholders involved in the design process and capturing business details. I'm pretty sure you can also do this with an org-mode doc and emailing the right questions to the right people, but discovery is hard and having a big post-it note brain storm with 30 people does sound like more fun!
I don't know, I feel like tech people are regularly amazed that other people can describe the events, processes, and data needs of their jobs if you just ask them. Likewise I grant that many people are poorly onboarded and end up "faking it to make it" in their careers. I've seen similar "people lighting up as they finally understand what the company does" moments through the use of literally any form of high-level modeling of the business. People are starved for context at most jobs.
Making telemetry opt-in would not have fixed this issue, the author is simply using this example to make an unrelated point about telemetry.
The real solution here is to remove all optional code from the critical path - this not only includes telemetry, but also things like in-app tutorials, automatic updates, crash reporting... If some code is not implementing a core function of the application, it should be allowed to crash and the application continue working. This is a core principle of good software engineering and it needs to be talked about more.
> “One of the most common misconceptions about microservices is that they solve a technology problem,” says Johnston. “But really they solve a business or organization scalability problem.”
I disagree. Service oriented architecture (note I don't say microservices, because I think that often goes too far) does solve technical problems. But they are problems that articles like this often ignore (maybe the authors have some misconceptions about microservices themselves).
For me the main benefit of seperate services isn't about organization of code or teams, it is about isolation.
The technical problems I think service seperation solves are:
How do I ensure that a security vulnerability in X doesn't compromise Y (least privilege)?
How do I minimize the blast radius if an unusual request or traffic patternconsumes an unexpected amount of resources (related to above, but doesn't necessarily need to be malicious)?
How do I deploy a change to X without having to rebuild and retest everything else?
How do I give a group of people access to manage X without giving them permissions for Y.
How do I optimize usage of some resources? (Note: multiple services have overhead, so for this to apply, you need to have enough scale that the benefits of fine-grained scaling and rightsizing of individual services outweighs the costs, but contrary to some comments you don't have to be quite as big as google for it to matter).
For some of these,"microservices" are not the only solution, and depending on your exact situation it may or may not be the best solution.
The important thing is to understand what specific problems you are trying to solve, why, and what the tradeoffs are.
I grew up a geek in a rural area, learning how to filter conversations for your audience was a critical skill.
Topics and conversational style amongst my D&D group was very different than amongst my other high school classmates and very different than when at my summer jobs.
This is just an article about people who never learned that skill.
I'm reminded of a few moments in my life that drove this home ...
A coworker (60 year old railway worker) who said "You shouldn't use ten dollar words in a twenty-five cent conversation."
A girlfriend who described the difference as "Weird is when you know you're weird and compensate when needed, strange is when you don't know you're strange."
2 minutes in, Paul types out [xfs] and then later [xtc] — what do you think these symbols are for?
My best guess from watching the video is that they are temporary place holders, breadcrumbs he leaves himself to return to, allowing him not to break his flow/train of thought. If that's the case, then the technique reminds me of Neil Strauss's "Power of the TK":
When you get suck on a word, or cant’ think of the right fact, what shouldn’t you do?
ImageOptim is fantastic. You can use it via CLI, so I've added it to a few different archiving / publishing workflows in the past. One thing that some people don't realize is you can run it multiple times on the same (PNG) images and get better results. Each filter is dependent on the input from the previous filter, so running them again can give you better results even after one run. This works out because filter 3 might give you some savings or rearrange the data in a way that filter 1 can now take advantage of.
The other JPG filters are lossless, but be aware that Guetzli is lossy!
I was impressed with just how much it could losslessly compress some massive JPGs until I did a visual diff. I can't see the difference, like one-bit deltas sort of thing, but it's not 1:1 lossless as I would expect.
First, a key limitation that every architect should pay attention. Redis reaches the limits of what you can do in well-written single-threaded C. One of those limits is that you really, really, *really* don't want to go outside of RAM. Think about what is stored, and be sure not to waste space. (It is surprisingly easy to leak memory.)
Second, another use case. Replication in Redis is cheap. If your data is small and latency is a concern (eg happened to me with an adserver), then you can locate read-only Redis replicas everywhere. The speed of querying off of your local machine is not to be underestimated.
And third, it is worth spending time mastering Redis data structures. For example suppose you have a dynamic leaderboard for an active game. A Redis sorted set will happily let you instantly display any page of that leaderboard, live, with 10 million players and tens of thousands of updates per second. There are a lot of features like that which will be just perfect for the right scenario.
I'm fond of Vale, where you can write your own rules or grab others off github. I wish we could crowd source a grammarly clone using it. Some efforts have been made to do so, but didn't get far.
After 20 years of programming, this is what comes to my mind when I hear "How do Design Programs":
1: Start with the data structure
The tools to operate on the data can be rewritten or improved without much pain. Changing the data structure is what hurts. If the data structure is elegant, the code will fall into place. If the data structure is messy, the code will grow into a giant hairy ball of madness.
2: Don't use a DB. Use human-readable text files
It makes everything so much easier. You can see if your data structure is elegant. Because then the files will be easy to read and navigate.
You will be able to start up your software 20 years from now without any installation and without having to go through the hell that is setting up the environment and updating dependencies. The website you are reading this on, Hacker News, stores all data in text files.
3: Build small tools to operate on the data
Like Linus started git by writing small tools "git-add", "git-commit", "git-log" etc. On the web you can have one url to do one thing (/urser, /item, /edit here on HN) and map them to one code file on your disk. If you want to use a framework like Django, Laravel or Express, those will unfortunately make it a bit harder to do that. But you can still stick somewhat closely to this approach.
4: Let people use it right away
If you are building it for yourself, you already have a user. Great. If not, let your target group use it right away. Otherwise, you will waste many years of your life building software features nobody understands or even wants.
I'll second the recommendation above. For folks coming to algorithms for the first time I would recommend _Grokking Algorithms
An illustrated guide for programmers and other curious people_ as a nice quick _tour_ of the field.
And the _Common Sense Guide_ (mentioned above) as the next book. For complete beginners this book has the best simple explanations of algorithms I've read in any book (and I have a whole bunch of them).
A major component of a strategy is constraint. In other words, what are you going to give up? If you're going to do All The Things, that isn't a strategy, it's a wishlist. You have to bring constraints into the picture before anything like a strategy emerges.
CPU utilization. Modern CPU can execute like 4-6 uops per cycle IIRC. Multiply that by clock freq and number of cores and you get theoretical max. Then take your executed instructions per sec and divide by this theoretical max. The better the ratio, the more efficient your program is.
Well said. Ultimately I think this is where much of the communication breakdown occurs when discussing Framework A vs Framework B online.
If you’re optimizing for _code_, sure. Stress about the ms it takes to load this or that thing. Client vs server. All of it is valid discussion from an engineering standpoint. Go nuts!
If you’re optimizing for _business_, then choose something that gets the job done. Fewer moving parts - and fewer resources needed to maintain it - the better.