What you say was true exactly only in most floating-point formats that were used before 1980.
In those old FP formats, the product of the smallest normalized and non-null FP number with the biggest normalized and non-infinite FP number was approximately equal to 1.
However in the IEEE standard for FP arithmetic, it was decided that overflows are more dangerous than underflows, so the range of numbers greater than 1 has been increased by diminishing the range of numbers smaller than 1.
With IEEE FP numbers, the product of the smallest and biggest non-null non-infinite numbers is no longer approximately 1, but it is approximately 4.
So there are more numbers greater than 1 than smaller than 1. For IEEE FP numbers, there are approximately as many numbers smaller than 2 as there are numbers greater than 2.
An extra complication appears when the underflow exception is masked. Then there is an additional set of numbers smaller than 1, the denormalized numbers. Those are not many enough to compensate the additional numbers bigger than 1, but with those the mid point is no longer at 2, but somewhere between 1 and 2, close to 1.5.
You are right, but only for a certain meaning of the word "geometry".
If "geometry" refers to the geometry of an affine space, i.e. a space of points, then indeed there is nothing special about any point that is chosen as the origin and no reason do desire lower tolerances for the coordinates of points close to the current origin.
Therefore for the coordinates of points in an affine space, using fixed-point numbers would be a better choice. There are also other quantities for which usually floating-point numbers are used, despite the fact that fixed-point numbers are preferable, e.g. angles and logarithms.
On the other hand, if you work with the vector space associated to an affine space, i.e. with the set of displacements from one point to another, then the origin is special, i.e. it corresponds with no displacement. For the components of a vector, floating-point numbers are normally the right representation.
So for the best results, one would need both fixed-point numbers and floating-point numbers in a computer.
These were provided in some early computers, but it is expensive to provide hardware for both, so eventually hardware execution units were provided only for floating-point numbers.
The reason is that fixed-point numbers can be implemented in software with a modest overhead, using operations with integer numbers. The overhead consists in implementing correct rounding, keeping track of the position of the fraction point and doing some extra shifting when multiplications or divisions are done.
In languages that allow the user to define custom types and that allow operator overloading and function overloading, like C++, it is possible to make the use of fixed-point numbers as simple as the use of the floating-point numbers.
Some programming languages, like Ada, have fixed-point numbers among the standard data types. Nevertheless, not all compilers for such programming languages include an implementation for fixed-point numbers that has a good performance.
Fixed point and Floating point are extremely similar, so most of the time you should just go with floats. If you start with a fixed type, reserve some bits for storing an explicit exponent and define a normalization scheme, you've recreated the core of IEEE floats. That also means we can go the other way and emulate (lower precision) fixed point by masking an appropriate number of LSBs in the significand to regain the constant density of fixed. You can treat floating point like fixed point in a log space for most purposes, ignoring some fiddly details about exponent boundaries.
And since they're essentially the same, there just aren't many situations where implementing your own fixed point is worth it. MCUs without FPUs are increasingly uncommon. Financial calculations seem to have converged on Decimal floating point. Floating point determinism is largely solved these days. Fixed point has better precision at a given width, but 53 vs 64 bits isn't much different for most applications. If you happen to regularly encounter situations where you need translation invariants across a huge range at a fixed (high) precision though, fixed point is probably more useful to you.
There are applications where the difference between fixed-point and floating-point numbers matters, i.e. the difference between having a limit for the absolute error or for the relative error.
The applications where the difference does not matter are those whose accuracy requirements are much less than provided by the numeric format that is used.
When using double-precision FP64 numbers, the rounding errors are frequently small enough to satisfy the requirements of an application, regardless if those requirements are specified as a relative error or as an absolute error.
In such cases, floating-point numbers must be used, because they are supported by the existing hardware.
But when an application has more strict requirements for the maximum absolute error, there are cases when it is preferable to use smaller fixed-point formats instead of bigger floating-point formats, especially when FP64 is not sufficient, so quadruple-precision floating-point numbers would be needed, for which there is only seldom hardware support, so they must be implemented in software anyway, preferably as double-double-precision numbers.
The core of my multiplayer arena game is in fixed point
I wanted absolute certainty that the rollback netcode would result in identical simulations on any platform, and integer math provides that. With set of wrapper functions and look up tables for trig it’s not that much worse than using regular floats
I am still uncertain if I actually would have been fine with floats, being diligent to round frequently and staying within true integer representable range… but now at least I’m far less afraid of game desyncs and it wasn’t that much work
Cross platform, cross USA games have been stable and fun to play, no fixed point complaints here
Floating point determinism has been a personal bugbear of mine for a number of years. You still have to be careful, but it's at the point where it's less work than switching to fixed point (cheap as that may be). There are even libraries [0] [1] that implement full reproducibility with negligible overhead. Compilers shipping incorrectly rounded stdlib functions remains an issue, but they're slowly improving. Language level support for float reproducibility is in the C++ pipeline, and already a design consideration on the Rust side. In a decade or so determinism issues might be a distant memory once you've ensured same inputs to the same instructions in the same order.
i.e. the difference between having a limit for the absolute error or for the relative error.
The masking procedure I mentioned gives uniform absolute error in floats, at the cost of lost precision in the significand. The trade-off between the two is really space and hence precision.
I'm not saying fixed point is never useful, just that it's a very situational technique these days to address specific issues rather than an alternative default. So if you aren't even doing numerical analysis (as most people don't), you should stick with floats.
Michael O. Rabin had important contributions in many domains, but from a practical point of view the most important are his contributions to cryptography.
After Ralph Merkle, Whitfield Diffie and Martin Hellman, Michael O. Rabin is the most important of the creators of public-key cryptography.
The RSA team (Ron Rivest, Adi Shamir and Leonard Adleman) is better known than Michael O. Rabin, but that is entirely due to marketing and advertising, because they founded a successful business.
In reality the RSA algorithm is superfluous and suboptimal. If the RSA team had never discovered this algorithm, that would have had a null impact on the practice of cryptography. Public-key cryptography would have been developed equally well, because the algorithms discovered by Merkle, Diffie, Hellman and Rabin are necessary and sufficient.
On the other hand, while without the publications of RSA, cryptography would have evolved pretty much in the same way, without the publications of Michael O. Rabin from the late seventies the development of public-key cryptography would have been delayed by some years, until someone else would have made the same discoveries.
Together with Ralph Merkle, Michael O. Rabin was the one who discovered the need for secure cryptographic hash functions, i.e. one-way hash functions, which are now critical for many applications, including digital signatures. Thus Rabin is the one who has shown how the previously proposed methods of digital signing must be used in practice. For example, the original signing algorithm proposed by RSA could trivially be broken and it became secure only in the modified form described by Rabin, i.e. with the use of a one-way hash function.
Originally, Merkle defined 2 conditions for one-way hash functions, of resistance to first preimage attacks and second preimage attacks, while Rabin defined 1 condition, of resistance to collision attacks. Soon after that it was realized that all 3 conditions are mandatory, so the 2 definitions, of Merkle and of Rabin, have been merged into the modern definition of such hash functions.
Unfortunately, both Merkle and Rabin have overlooked a 4th condition, of resistance to length extension attacks. This should have always been included in the definition of secure hash functions.
Because this 4th condition was omitted, the US Secure Hash Algorithm Standards defined algorithms that lack this property, which has forced many applications to use workarounds, like the HMAC algorithm, which for many years have wasted time and energy wherever encrypted communications were used, until more efficient authentication methods have been standardized, which do not use one-way hash functions, for instance GCM, which is today the most frequently used authentication algorithm on the Internet.
I think you're vastly underplaying the importance of RSA to cryptography. Personally it was the first time I was exposed to the concept of public key cryptography (in the 1980's). "would have been delayed by some years" is very dismissive. The same thing can be said of many inventions. Yet someone is/was the inventor.
RSA were the first to provide a practical and easy to understand implementation and that had a huge impact in practice.
That's not to downplay Rabin's or others contribution. That RSA pursued a certain commercial strategy that you may or may not like is not really relevant.
They didn't really found a successful business. They founded a middling business that didn't do much but license a patent until Security Dynamics, a smart card company, bought them and took over the name.
The story that I remember going around is that they each made some millions of dollars. That was a lot of money at the time for academics. I audited a cryptography course given by Adi Shamir in the early 90's and you couldn't tell he was rich though.
On the contrary, you cannot find frequently descriptions about the role of Michael O. Rabin in the creation of public-key cryptography, so few people are aware of it and I bet that no AI model can generate any text even remotely resembling this, because this information cannot be found in any single place in the possible training texts.
You can find definitions of secure hash functions everywhere, but pretty much nowhere you will find who are the authors of the conditions that are used in the modern definition and who have introduced the use of one-way hash functions.
I did not find this information anywhere, before reading the original publications of Rabin and Merkle from 1978/1979 and some later follow-up papers written by them.
You will not find this historical information in Wikipedia and I believe that it is important to know who are the true authors of the things that one uses daily. Connecting to this site or to any other site with https uses digital signatures that depend on the collision-resistant hash functions defined by Rabin and Merkle.
The Wikipedia article about Michael O. Rabin lists many of his achievements, but all those that are listed there are much less important than his contribution to the definition of the one-way hash functions, which lead to secure digital signatures.
Wikipedia mentions only the Rabin signature algorithm, but that has negligible importance, because it has been used only very rarely. On the other hand all other signature algorithms are based on the work of Rabin, by using secure hash functions.
I wouldn’t really call that AI slop. Some people just write longer posts because they’ve got a lot they want to get across, and you can usually tell it reflects their own opinions and what they think matters in the discussion. Actual AI-generated stuff tends to come off more generic and lacks that personal angle.
I don't think that is AI slop. adrian_b often post long posts because he thinks he has a lot to say, but you can often tell that they contain his personal views and points that he thinks are important related to the discussions whereas actual AI slop tends to be bland and generic.
Modern flash memory, with multi-bit cells, indeed requires more power for reading than DRAM, for the same amount of data.
However, for old-style 1-bit per cell flash memory I do not see any reason for differences in power consumption for reading.
Different array designs and sense amplifier designs and CMOS fabrication processes can result in different power consumptions, but similar techniques can be applied to both kinds of memories for reducing the power consumption.
Of course, storing only 1 bit per cell instead of 3 or 4 reduces a lot the density and cost advantages of flash memory, but what remains may still be enough for what inference needs.
The basic physics of reading from Flash vs. DRAM are broadly similar, and it's true that reading from SLC flash is a bit cheaper, but you'll still need way higher voltages and reading times to read from flash compared to DRAM. It's not really the same.
Those were originally Philips devices, but like with the American JEDEC part names, after a device with an European part name, like BC337 was registered, any semiconductor device manufacturer could sell equivalent devices.
The European part numbers provided much more information than the American part numbers.
JEDEC 2Nxxxx just told you that this is some kind of transistor or thyristor, instead of being a diode like 1Nxxxx.
BC told you that this is a silicon small-power audio-frequency transistor.
There were separate codes for other materials and for many other kinds of transistors, diodes and thyristors (for example AD = germanium high-power audio-frequency transistor, BF/BL = Si low/high-power RF transistors, BS/BU = Si low/high-power switching transistors, BR/BT = Si low/high-power thyristors, BA/BY = Si low/high-power rectifiers, BB = Si varicaps, and many others).
Motorola and some other US companies, like Texas Instruments and Fairchild, entered the transistor market very early, when they defined types like 2N2222, which became industry standards.
However, because these devices were defined early, they had rather poor characteristics. When European companies like Philips, Siemens, Thomson, SGS-ATES entered the market later, they defined transistors and other devices with improved characteristics.
Because of this, in Europe the devices with European part numbers, like BC337, were generally preferred, because they provided better analog performance, e.g. lower noise and higher bandwidth.
However nowadays this has become mostly irrelevant, because a legacy transistor vendor makes only a small number of different kinds of transistors, distinguished mainly by die size, because bigger sizes are needed to handle bigger currents. Then the transistors are packaged and marked with any of the legacy part numbers, depending on what part number the customer orders.
So while old transistors may have quite different characteristics depending on the part name, many modern transistors behave the same, regardless how they are marked.
> BC told you that this is a silicon small-power audio-frequency transistor.
BC breaks down as a silicon device, with no heater voltage, and a "triode".
If it was germanium, it would be AC <something>.
So BC548 is a silicon "triode", AC128 is a germanium "triode", and PC97 is a triode with a 300mA-rated heater (P is series connected with other valves, 300mA) in a B7G base (the 9).
"BF" might be an RF transistor although "F" was really used to mean a pentode in valves.
And those dual NPNs used in expo converters in synths might be accurately enough labeled as BCC548, similar to the ubiquitous ECC83 dual triode.
You also see this with diodes, were AA119 is a germanium small-signal diode, and BY127 is a silicon high(-ish) power rectifier diode, for example.
No. "B" is just the material (for instance "A" = Ge, "B" = Si, "C" = GaAs and related materials).
The letter that encodes the semiconductor material replaced the letter that encoded the voltage or the current used by the heating filament of vacuum tubes.
The material letter has nothing to do with the kind of device.
Examples of silicon diodes: BA (small power rectifier), BB (varicap), BY (high power rectifier), BZ (Zener diode).
It is true that some of the letters that denote kinds of devices have been inherited from the previous nomenclature of European vacuum tubes.
So C, D, F, L, used for low-power/high-power AF/RF bipolar transistors come from the letters used for low-power/high-power triodes/pentodes.
However other letters, like S, U, R, T, used for switching transistors and thyristors (a.k.a. SCRs), were new for the semiconductor device nomenclature.
While power consumption and speed are indeed the advantages, depending on the concrete application it may be very difficult or even impossible to make an optical computing device with a size comparable with an electronic device.
The intrinsic size of the optical computing elements is much larger, being limited by wavelength. Then a lot of additional devices are needed, for conversion between electrical and optical signals and for thermal management.
Optical computing elements can be advantageous only in the applications where electronic devices need many metallic interconnections that occupy a lot of space, while in the optical devices all those signals can pass through a layer of free space, without interfering with each other when they cross.
This kind of structure may appear when doing tensor multiplication, so there are indeed chances that optical computing could be used for AI inference.
Nevertheless, optical computing is unlikely to ever be competitive in implementing general-purpose computers. Optical computers may appear but they will be restricted to some niche applications. AI inference might be the only one that has become widespread enough to motivate R&D efforts in this direction.
Bold claim to say these challenges will never be surmounted. Either a more-economic technology would have to mature first, or civilisation halt progress for that to be true. If scientific advances could yield miniaturised photonics that offer a significant cost/benefit over any contemporary technology the concept will still be pursued. Unless you are suggesting that it is theoretically and physically impossible?
The dust that comes from meteorites contains up to 4 fractions: silicates, which cannot be oxidized, metallic iron, which oxidizes, but it does not form volatile substances that can be smelled, hydrocarbons in the form of a tar or pitch, which can burn but it cannot be ignited easily, and finally a fraction made of iron sulfide (troilite) with small quantities of other sulfides.
In contact with air, the sulfides will be oxidized, releasing sulfur dioxide. Burning black powder also releases sulfur dioxide, which is the main reason for its smell. Burning pure sulfur will produce the same smell.
Any dust on the Moon still consists mostly of silicates which cannot be oxidized.
When dust comes from meteorites, it contains a fraction made of iron sulfide (with small quantities of other sulfides) and another fraction made mainly of hydrocarbons.
The metallic sulfides can be oxidized, but they will not burn violently. The hydrocarbons are like a tar or pitch, because the volatile hydrocarbons would have sublimated in vacuum. So neither that tar is easily flammable.
The gunpowder smell is likely to be caused by the oxidation of the sulfides from the dust, which releases sulfur dioxide, the same like burnt gunpowder.
CUDA variants extend several programming languages, including C, C++ and Fortran.
None of the extended languages is the same as the same as the base language, in the same way like OpenMP C++ in not the same as C++ or OpenMP Fortran is not the same as Fortran or SYCL is not the same as C++.
The extended languages include both extensions and restrictions of the base language. In the part of a program that will run on a GPU you can do things that cannot be done in the base language, but there also parts of the base language, e.g. of C++, which are forbidden.
All these extended languages have the advantage that you can write in a single source file a complete multithreaded program, which has parts running concurrently on a CPU and part running concurrently on a GPU, but for the best results you must know the rules that apply to the language accepted by each of them. It is possible to write program that run without modification on either a CPU or a GPU, but this is paid by a lower performance on any of them, because such a program uses only generic language features that work on any of them, instead of taking advantage of specific features.
"Hot" companies with stupid managers often have such workdays.
In the case of engineers and programmers, the amount of useful completed "work" has only a very weak correlation with the length of the workdays.
Good engineers or programmers will think anyway most of the time about the problems that they must currently solve, regardless whether they are in the office or at home or in any other place, and regardless whether to an external observer they appear to be "working" or they appear to do nothing.
Programmers who spend all day typing lines of code into a computer, are more likely to not be competent programmers, because otherwise they would have found ways to automate such activities that require continuous physical involvement, making impossible the allocation of enough time for thinking about the right solution.
If whatever they do does not require true thinking, then that is the kind of job that can be done by AI agents.
In those old FP formats, the product of the smallest normalized and non-null FP number with the biggest normalized and non-infinite FP number was approximately equal to 1.
However in the IEEE standard for FP arithmetic, it was decided that overflows are more dangerous than underflows, so the range of numbers greater than 1 has been increased by diminishing the range of numbers smaller than 1.
With IEEE FP numbers, the product of the smallest and biggest non-null non-infinite numbers is no longer approximately 1, but it is approximately 4.
So there are more numbers greater than 1 than smaller than 1. For IEEE FP numbers, there are approximately as many numbers smaller than 2 as there are numbers greater than 2.
An extra complication appears when the underflow exception is masked. Then there is an additional set of numbers smaller than 1, the denormalized numbers. Those are not many enough to compensate the additional numbers bigger than 1, but with those the mid point is no longer at 2, but somewhere between 1 and 2, close to 1.5.
reply