PSA: Please be cautious because this is an excellent opportunity for taking over packages and injecting malware by malicious people.
Example: https://www.npmjs.com/package/duplexer3 which has 4M monthly downloads just reappeared, published by a fresh npm user. They published another two versions since then, so it's possible they've initially republished unchanged package, but now are messing with the code.
I'm not saying it's a malicious attempt, but it might be and it very much looks like. Be cautious as you might don't notice if some packages your code is dependent on were republished with a malicious code. It might take some time for NPM to sort this out and restore original packages.
I just tested, and it definitely looks like a troll / hack.
> duplexer3@1.0.1 install /Users/foo/Code/foo/node_modules/duplexer3
> echo "To every thing there is a season, and a time to every purpose under the heaven:
A time to be born, and a time to die; a time to plant, and a time to pluck up that which is planted;
A time to kill, and a time to heal; a time to break down, and a time to build up;
A time to weep, and a time to laugh; a time to mourn, and a time to dance;
A time to cast away stones, and a time to gather stones together; a time to embrace, and a time to refrain from embracing;
A time to get, and a time to lose; a time to keep, and a time to cast away;
A time to rend, and a time to sew; a time to keep silence, and a time to speak;
A time to love, and a time to hate; a time of war, and a time of peace.
A time to make use of duplexer3, and a time to be without duplexer3."
To every thing there is a season, and a time to every purpose under the heaven:
A time to be born, and a time to die; a time to plant, and a time to pluck up that which is planted;
A time to kill, and a time to heal; a time to break down, and a time to build up;
A time to weep, and a time to laugh; a time to mourn, and a time to dance;
A time to cast away stones, and a time to gather stones together; a time to embrace, and a time to refrain from embracing;
A time to get, and a time to lose; a time to keep, and a time to cast away;
A time to rend, and a time to sew; a time to keep silence, and a time to speak;
A time to love, and a time to hate; a time of war, and a time of peace.
A time to make use of duplexer3, and a time to be without duplexer3.
Not to mention, it’s a Pete Seeger song, the byrds just covered it. I may be wrong but I think Seeger wrote it for Judy Collins to sing.
Edit: ok nope, Seeger didn’t “write” it for Collins, she’s just another one to cover it. Here they are both doing it if you’re interested: https://youtu.be/fA9e-vWjWpw
Start posting large parts of, say, the New International Version, let me know how that goes for you.
IOW, unless it’s the King James, it is likely very much subject to take down notices. Though I’m guessing a malicious troll is much more likely to know The Byrds than they are Old Testament.
And all this is happening just as after the public release of a serious exploit which allows malicious code to do all sorts of nefarious things when it is somehow installed on the target machine. Hmm.
Given that there's hints, at least, that the problems were caused by some particular developer's actions, I wonder about the security model for package-managed platforms altogether now. If I were a big cybercrime ring, the first thing I'd do would be, get a bunch of thugs together and knock on the front door of a developer of a widely-used package; "help us launch [the sort of attack we're seeing here] or we'll [be very upset with you] with this wrench." Is there a valid defense for a platform whose security relies on the unanimous cooperation of a widely-scattered developer base?
With cases like the current one, or the leftpad incident in 2016, I'm surprised package registries still allow recycling old package names after a package was deleted. Really seems like deleted packages should be frozen forever - if the original author never recreates it or transfers ownership, then people would have to explicitly choose to move to some new fork with a new id.
But your point about pressuring or bribing package authors still stands as a scary issue. Similar things have already happened: for example, Kite quietly buying code-editor plugins from their original authors and then adding code some consider spyware (see https://news.ycombinator.com/item?id=14902630). I believe there were cases where a similar thing happened with some Chrome extensions too...
> With cases like the current one, or the leftpad incident in 2016, I'm surprised package registries still allow recycling old package names after a package was deleted.
CPAN requires the old author to explicitly transfer or mark it abandoned-and-available-to-new-owner.
For all the things wrong with perl5 (and I love it dearly but have spent enough time I can probably list more things wrong with it than the people who hate it ;) it's always a trifle depressing to watch other ecosystems failing to steal the things we got right.
This happens all the time. The new generation creates something cool because what our parents created isnt cool any more, only to fail exactly on the same spot as our parents. Only, it was already solved in the parents last version.
This goes for cloth design, cars, houses, kitchen wares and so on, as well as software.
Just look at the microwave oven earlier...
Modern microwave ovens have all adopted impractical and quirky new UIs, when the old concept of knobs was simple and worked fairly well in the first place.
My oldest one was just two dials, the second one, 15 years old had loads of buttons and stuff, really stupidly spread out, you had to press watt, minutes, seconds, start and start was not in a corner, not in top/bottom row or any other logical place so you had to search it every time. I glued a rubber piece to it so I could find it again without having to bend down and search.
Since then I have made sure the microwave has two dials, one for time, one for effect.
Remember the watercooker that had just an on & off switch?
Then came one with an option button for 80 or 100 degrees (176 or 212, in freedoms). Never knew I needed that, but that just changed my live and I can not do without it. Reason: 80 degrees water is hot enough for my needs and saves time.
Our latest has 3 buttons, with different possiblities, beebs like a maniac when ready (an option which is not unset-able) and can do things I never knew anyone would need (like keeping it at x degrees for y minutes).
I guess it is like evolution: you experiment, keep what works and get rid of all things unfit.
Packages / projects being frozen. AFAIR that's how SourceForge works/worked. I remember a few years back being baffled that I couldn't delete my own project.
But it makes sense, other projects might depend on it, so it's archived.
It's just npm that's broken. I've never used a package manager for any other language that had these kinds of issues. It's exacerbated by the massive over-reliance on external packages in JS too. `left-pad` really shone a light on how dependencies in JS land really are brought in without too much thought.
Although, I've never considered this in the case of an actual attack. It would make sense to actually fingerprint the entire source tree and record this too somewhere, so when you build it you know you are getting the right thing. Teapot basically defers this to git.
> Is there a valid defense for a platform whose security relies on the unanimous cooperation of a widely-scattered developer base?
The defense is staged deployment and active users. This obviously depends on the blutness of the malicious code.
If I may assume easily noticed effects of the malicious code: A dev at our place - using java with maven - would update the library, his workstation would get owned. This could have impacts, but if we notice, we'd wipe that workstation, re-image from backup and get in contact with sonatype to kill that version. This version would never touch staging, the last step before prod.
If we don't notice on the workstation, there's a good chance we or our IDS would notice trouble either on our testing servers or our staging servers, since especially staging is similar to prod and subject to load tests similar to prod load. Once we're there, it's back to bug reports with the library and contact with sonatype to handle that version.
If we can't notice the malicious code at all until due to really really smart activation mechanisms... well then we're in NSA conspiracy land again.
> If we can't notice the malicious code at all until due to really really smart activation mechanisms... well then we're in NSA conspiracy land again.
What about really dumb activation methods? I.e., a condition that only triggers malicious behavior several months after the date the package was subverted. You don’t have to be the NSA to write that.
What’s scary here is that there are simpleminded attacks that, AFAIK, we don’t know how to defend against.
Mh, I have a rather aggressive stance on these kind of incidents, no matter if they are availability or security related. You can fish for them, you can test for them, and there are entire classes of malicious code you cannot find. For everything you do, turing complete code can circumvent it. There's a lot of interesting reading material going on in the space of malware analysis regarding sandbox detection, for example.
So stop worrying. Try to catch as much as feasible before prod. Then focus on detecting, alerting and ending the actual incident. If code causes an incident, it't probably measurable and detectable. And even then you won't be able to catch everything. As long as a server has behavior observable from the internet, it could be exfiltrating data.
Tested restores with at most 59 minutes of data loss for prod clusters within 90 minutes after order. 30ish minutes of downtime. We could even inspect binlogs for a full restore afterwards on a per-request basis for our big customers.
Sounds like good hygiene, though it seems burdensome if everyone must do it or seriously risk infection. Ideally there would be at least minimal sanity checks and a formal process before a package can be claimed by someone else.
In case anyone was considering sending him $10, no, his hypothetical code would not be running on the Google login page. Google does not pull in external dependencies willy nilly like that.
On Google scale you quite certainly want to do that. Not just for security, but for legal reasons. You really don't want to end up using for example AGPL licensed stuff in wrong places and if you just blindly pull stuff with dependencies from package manager, this could easily happen.
One of the recent True Geordie podcasts features the "YouPorn Guy" who talks about finding it near impossible to get lawyers not on a retainer from Google to fight them.
Sure a legal audit is standard and usually much simpler than a full source audit for security, which has a complexity proportional to the project size.
I sincerely hope all modern package managers, when invoked with sudo, immediately spawn a very-low-privilege process that does most of the work sandboxed to /tmp/whatnot, and the root process just copies files to the right place and runs some system calls to update databases etc.
Most package managers I know support Turing complete install hooks. How would a package manager detect what parts of those require/are safe to run with root?
No, that's the entire point. They need almost nothing at all but the ability to run code fast in a loop with memory calls. The entire point is that they bypass privilege checks.
I'm not sure if it would help much. That means you either have to have users be able to recognize and eyeball-validate hashes ("sure, this is left-pad-5ffc991e; that's what I want! Wait, shit, it's actually left-pad-5ffd991e, never mind; wrong package), or you need pre-existing databases of trusted hashes (which either puts you right back at a registry a la NPM, or leaves you reliant on a package.lock file or similar, which doesn't cover many common use cases for secure package signing).
Detailed description what you could do with a malicious npm package is currently on he front page: "Harvesting credit card numbers and passwords from websites"
Hey, I wrote that article :) - yes it was pure coincidence, I just decided with all the security stuff going on this week (Spectre/Meltdown, I hadn't heard about the npm stuff) I'd write and article about it.
I am very surprised that a package manager of this calibre and impact abstains from best practices when it comes to authentication through code-signing. Other package managers are miles ahead of NPM. For example, Nix, which uses immutability and hashing to always produce the same artifact, regardless of changes of the sources.
So I know rpms and debs are signed, as I've setup repos for both. Docker repositories require a valid SSL key (or you have to manually allow untrusted repos). But do Python packages and Ruby gems have signature verification? How does pypy/pip and gem deal with validating a package is what it claims to be?
You have a point, but we need to take into account that the technology has been around for a long time, the risks are well known and documented, and safety concerns of most of these package managers have been addressed to maintainers.
The example in the article has come to light accidentally, but we must seriously ask ourselves how many incidents are currently unidentified.
Besides, you can use Nix for 'normal' development. It is suitable for more things than just a distro package manager.
Signing won't help unless the end user specifies the signature or certificate that they expect (signing would only help ensure package upgrades are from the same author).
If you're going to have clients specify a signature anyway, then you don't need to sign packages, you just need strong one way hash function, like SHA-1024 or something. User executes "pkg-mgr install [package name] ae36f862..."
Either way, every tutorial using npm will become invalid.
"npm install packagename" could record the public key in package.json (or package-lock.json) on first save, and only accept installs (or upgrades) matching the same public key. Just like how android app code signing works, or similar to ssh known_hosts trust-on-first-use.
Granted it wouldn't save those adding a new package to a project the first time, but it would save the bacon of anyone re-running "npm install" in an existing project, for example during a deploy, or when trying to upgrade to a newer version of a given package.
independent site that maps packages to author certs that npm uses for verification at install time?
also, this is a problem that every package mgmt system faces. they alert on changes on upgrade but there's a requirement at the end user level to verify that at install time, the cert being trusted is the right one.
I'm surprised there wasn't a global lock-down on new package registrations (or at least with the known names of lost packages) while they were working to restore them.
How does RubyGems handle a package being removed and replaced by a different (and maybe malicious) actor? Not allow a package to be deleted? Block the package name from being claimed by someone else?
> Once you've yanked all versions of a gem, anyone can push onto that same gem namespace and effectively take it over. This way, we kind of automate the process of taking over old gem namespaces.
Shit. That's a good point, I downloaded the Heroku CALI during the attack and it uses duplexer3. I got a weird message that seemed "off" during postinstall.
This is truly great as proof of concept and also as a reminder that in software development building 80% of perceived* functionality usually takes only tiny fraction of total development time which is required to build final product.
* - this app looks and feels like “almost complete spreadsheet” yet it provides much less than 1% features of even a basic spreadsheet.
And 80% of perceived functionality is what most users — even people who rely on spreadsheets for their work — need.
I'm not saying including the other 20% of functionality isn't a good thing to do, but when you're creating a new product or open source project, you can get by (and thrive) building only the 20% of functionality that's used 80% of the time.
Why? Because 1% of functionality is still better than 0% (never shipping).
You're not Microsoft. You don't have to keep your dominance in the spreadsheet software market like Microsoft has to.
It doesn't actually work well. It doesn't do basic stuff, which makes it entirely unusable. Like you can't move up/down/left/right using the arrow keys. You can't save it. It crashes if you make basic mistakes.
The 80% of perceived functionality is that it looks like a spreadsheet app and actually does some basic calculations.
It actually does about 5% of what you'd expect even the most basic of spreadsheets to do. And all the polish of even the most basic functionality is missing, which is what takes most of the time.
Which is why you should always avoid putting a pretty looking but barely functional mockup in front of the pointy hairs. They then think it's almost done.
> And 80% of perceived functionality is what most users — even people who rely on spreadsheets for their work — need
This is exactly the insidious myth that the GP is pointing out.
Yes, 80% of the "perceived" functionality might be 1% of the work. But the remaining 20% is a long tail of utterly essential features that are completely fragmented between users. Each user only cares about 80.1% of features, but without the extra 0.1% your spreadsheet is useless.
One user will care that it doesn't print. Another will care that it can't do a sum over columns. Another will care that it doesn't have charts. Another will care that you can't make a bold heading. Another will want statistics functions. Basically your "80% of functionality" has zero value until you do years of work to implement all those 0.1% features.
This myth leads countless people to implement what they think is MVP and then be mystified that their insanely great work still has zero value to anybody they actually show it too.
The problem is that the total economic value (as measured by market price) for the users who only use that 80% of perceived functionality is close to 0 (google docs and OO/LO are free)
The incredibly difficult 20% is what people pay for. It's what the businesses that shell out billions of dollars a year pay for. And it's what a successful product in the space needs to tackle
On the other hand, you are taking what appears to be a general observation and applying it to this specific instance. Sure, there are plenty of free spreadsheets available. But what about other areas? There's plenty of areas still where a well executed easy to use minimal version of a product can compete in the existing for-pay ecosystem, and as we are discussing, minimal can pay off since 20% of functionality may take 5% or less of the time it would take to do 100%.
No, I'm specifically just saying that not everyone's "final product" is the same thing, and depending on the target market and audience, a minimal product may be worthwhile, and the economic value isn't always "close to 0", as the parent I replied to stated.
Also, I don't think that "feature minimalism is a good design philosophy" is always true, even though it may guide you well in most cases.
Um no. Google docs is free for the same reason IE was free. The economic value is tremendous, such that the creator is willing to spend billions on it.
Really? The first thing I did was select a cell, type =, then press the arrow keys. Nothing happened.
The perception of being a "almost complete spreadsheet" might be due to the UI looking similar to older versions of Excel, leading people to assume stuff is there that doesn't.
Also, I wonder how much of this code would survive if you want to get beyond this 1%. Probably nothing, which makes this a road with a dead end. This doesn't mean it is not a useful and brilliant demonstration of cool language features (hacks?), which is how I see it. But there's no relationship between this amount of work/code/lines for this percentage and the amount of work/code/lines for any reasonably higher percentage of spreadsheet functionality.
Protip: If you use paid email, ask your email provider how to hide your IP while sending via SMTP. I asked mine and it turned out that they run a separate SMTP port which you can use and then your IP won't be included in message headers. This is a fairly popular provider, but I won't mention their name as they don't advertise this feature and provide it on request.
Yup, my provider is quite easy to find, I don't mind. I just didn't want to make this info to be googleable with their name, to respect that they never publicly wrote about it.
No idea why they don't advertise this feature. They wrote that they only tell about it to people that are actually having a problem about it. I guess it might be easier for them or for other email-ops to handle abuse/spam issues when this header is available?
> The app doesn't auto-update favorites yet. For now you can manually update via pull-to-refresh.
Just to let you know: I favorited a song (using menu bar) and then pulled to refresh in my favorites view. The song was shown twice on the list. Same after another refreshes. After restarting the app the song is shown only once, but after pulling to refresh it's doubled again. The second copy appears slightly after the first, so it's probably some list populating launched twice.
The app is awesome and I loved it at the first sight. Thank you! I didn't know Hype Machine before, but they seem to serve some good music.
You should provide links to iTunes & Amazon to allow easy purchases if someone would like to add songs to their music library.
This is a summary of my understanding of their method after a very quick scan of their docs:
They assume working as a malicious/selfish pool having less than 50% of hash rate, but still a significant portion of the total hash rate. All other miners that are not part of the selfish pool are called honest miners.
When selfish pool finds a block, they don't advertise it but continue mining their forked, private blockchain. They have an advantage of one block over the public blockchain now. Of course they have no chance of building longer blockchain in the long term, as they have less than 50% of hashing power and the public blockchain will always get longer after some number of blocks. But what they count on is this:
Scenario 1: honest miners discover a block and the public blockchain gets the same length as the selfish blockchain. They immediately publish their block as soon as they discover someone else discovered a block. They hope to create a race condition and a public blockchain fork - so that some hones miners will get the “honest” block, but some of honest miners will get their “selfish” block and start mining using it as a base. Having some of the honest miners on their side they have a chance that their fork will get longer and the “honest” fork will be declined by the network.
Scenario 2: selfish pool is lucky and discovers another block, giving their blockchain two blocks advantage over the public blockchain. They continue mining and they publish one block for every block discovered by the honest miners. This creates race condition with some of the honest miners on their side, but they still have some blocks found and not published. They publish all their remaining blocks as soon as their advantage decreases to one block. The network chooses their branch as it's longer and they get all the reward coins from their secretly mined chain.
Now, I know nothing about blocks discovery/notification mechanisms over the network and how fast it works, so an important question to someone knowledgeable is if this is a probable scenario that their block published only after some competing block has been found and published has still a chance to get to some significant number of honest miners first so that they start mining over their block - as this is required for their strategy to work.
If the above is viable, then this strategy of course requires some significant hash rate share, but I remember that even having 10% of total hash rate, the probability that you will mine couple of blocks in a row is quite high - and that's all you need to create situations when you have two-three blocks advantage over the public blockchain.
In scenario 1 the selfish pool will lose the block in most cases. Modern pools have a lot of connection to the network, so as not to waste mining on orphaned block. It will get the honest block before the selfish pool block.
This is interesting since I have found very little information about Bitcoin operating on poorly connected networks (i.e. most nodes are not directly reachable from any given node.) Given the current trend toward internet censorship, I wonder if Bitcoin will have to adapt to such networks as the norm.
This is nice if considered a programming exercise but it's actually pretty useless for real world use.
This data is static, that is it very rarely changes. The whole dataset is also very tiny. It's just better to store this data in-house and don't provide unnecessary point of potential failure by using an external service.
An idea on how to make it more useful is to provide downloadable data dump in several formats suited for immediate import into various data stores. An extra addition might be modules for popular frameworks providing access to this (local) data in all the ways your web service does.
I'm not trying to diss, just trying to find some value for real world usage. The service is neat otherwise.
>> It's just better to store this data in-house and don't provide unnecessary point of potential failure by using an external service.
Isn't this the point of rfc2616 sec13? I haven't looked, but if there are etags etc then I think a computer is better than a human at looking for new data, no?
totally get your point. although there are cases when using an external service is actually more convenient. maybe riskier but still convenient. thanks.
Current Matchbook offer seems quite humble. Publishers will be naturally reluctant to sell their $22 books for $2 but I hope they'll understand very soon that this isn't actually a discounted sale. It's either extracting more money from past sales or making future sales more likely to happen (and also extracting more money from them).
Thanks for the list. I was trying to find it as well.
Serious question: what would sway one to register and use one of the above extensions compared to what's already out there? Other than localized content, I can't imagine typing شبكة would be simple...
I have a hard time comprehending your question. They've added a bunch of non-English TLDs which are clearly for use by non-English sites, but when you ask why someone would use them, you exclude the obvious use-case out of hand.
Aside from communicating with computers and other humans, what would anyone use the internet for?
Hmm. On my (US) Mac, switch input method to Arabic, hit af;m and get شبكة . So no harder to type than "info" really if you already have your machine set up to switch between Arabic and Roman/Latin keyboard layouts.
It is somewhat ironic, that none of the registries are in (one of) the language's home countries... most especially given the recent NSA things happening.
At today's Apple, both software and hardware are means to sell 3rd party software. They earn so much on their ecosystem that it's more profitable to give OS updates for free and provide more unified and less fragmented ecosystem to both users and 3rd party developers.
Lowering prices for next generations of their hardware is also a sign that they really want to put their foot in as many users' doors as possible. I think they rarely lowered hardware prices just couple years ago, before they had this massive revenue stream called App Store.
![http://www.foreignlanguagekeyboard.com/images/kbarabic[1].jp...](http://www.foreignlanguagekeyboard.com/images/kbarabic[1].jpg){kind=link}
Example: https://www.npmjs.com/package/duplexer3 which has 4M monthly downloads just reappeared, published by a fresh npm user. They published another two versions since then, so it's possible they've initially republished unchanged package, but now are messing with the code.
Previously the package belonged to someone else: https://webcache.googleusercontent.com/search?q=cache:oDbrgP...
I'm not saying it's a malicious attempt, but it might be and it very much looks like. Be cautious as you might don't notice if some packages your code is dependent on were republished with a malicious code. It might take some time for NPM to sort this out and restore original packages.