For numerical optimization, a couple good textbooks are:
- "Practical Optimization" by P. E. Gill, W. Murray and M. H. Wright: a little old (1982), but provides a solid foundation
- "Convex Optimization" by S. Boyd and L. Vandenberghe: the standard for learning convex optimization (also available as a free PDF from the author's website)
- "Convex Analysis and Monotone Operator Theory in Hilbert Spaces" by H. H. Bauschke and P. L. Combettes: covers a more specialized area of numerical optimization, but the notation is beautiful (IMO) and it acts as a useful reference for recent research on, e.g., operator splitting methods
Nonconvex optimization doesn't have the same depth of theoretical underpinnings or canonical body of knowledge as convex optimization so I don't imagine there's a textbook on it that would be authoritative. In the universe of optimization, convex optimization is a special case (linear optimization in turn is a special case of convex); non-convex optimization is everything else!
It's kind of like convex optimization is English, and nonconvex optimization is non-English. I'm not sure it's possible to write a text on non-English.
That doesn't non-convex optimization problems are unsolvable, merely that there are many different attacks that aren't necessarily coherently linked. A few common ones include:
a) convex reformulation, where possible.
b) partitioning into convex regions (used in global optimization)
c) heuristic/evolutionary approaches
d) specialized approaches for particular problem structures like integer programs, complementarity problems etc. (there are good textbooks for these)
There are a few good surveys of the landscape however. Most are journal pubs. This text [1] seems to be a good one.
The topics covered are fairly broad and overall it seems like a nice collection of notebooks for teaching. Also, I agree with the choice to use Anaconda to install the dependencies. In my experience teaching similar type workshops (to engineering undergrad and grad students), Anaconda provides a good balance of simplicity and coverage, particularly with audiences of varying backgrounds.
The chart https://twitter.com/FatTailCapital/status/899714838796148745 was produced as described in the footnote (mining hashrate × efficiency linearly declining from 1.5 to 0.2 J/GH), which IMHO fairly represents the "upper bound" of the energy consumption. The real consumption is likely lower. For example my study estimates the current efficiency is between 0.100 and 0.195 J/GH.
To give you an idea, "1M US homes continuously" is roughly the annual electricity consumption of decorative Christmas lights in the US.
This paper appears to be from 1998 [0]. No judgment on its quality; I'm just trying to provide a reference for other readers of the post.
[0]: A.C.C. Coolen, in ‘Concepts for Neural Networks - A Survey’ (Springer 1998; eds. L.J. Landau and J.G. Taylor), 13-70
‘A Beginner’s Guide to the Mathematics of Neural Networks’
They have a datasheet [0] for an "Environmental Monitoring Sensor" which lists a line-of-sight range of 50 miles and 1–3 miles in urban settings. For comparison, Digi's XBee PRO ZigBee wireless modules have a line-of-sight range of 2 miles [1]. I'm a little bit skeptical about the 50 mile range claim (seems too good to be true), but this is an interesting product nonetheless.
> Our team at Beep Networks is now working with LoRa radios here in San Francisco, and we’re getting signals through at over a mile of range. That’s in the city, through walls, with a tiny battery-powered sensor device — no towers or giant antennas involved. We know folks who are getting 10 miles in every direction when they put these radios on towers in rural areas, where there’s less interference.
- "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks"
- https://arxiv.org/abs/1803.03635