Thread For Your Useless Discoveries

hkoenig · Post by **hkoenig** » October 2nd, 2020, 9:50 pm

re: Microsoft development practices--That matches my experience.

Starting in the last century I did contracting at Microsoft, off and on, for about ten years, converting low level Windows code (registry management, video codecs, DRM, Silverlight, etc.) to run in the MacOS environment. Nobody wrote understandable or debuggable code, as far as I could tell. I think they were penalized for every comment or any extra whitespace character or any variable name longer than three characters, and lost stock options for producing actual documentation or design specs ("the source is the documentation" I was told once). The MacOS development system back then allowed only 10,000 compiler warnings before giving up completely, and at least once I had to work around that limitation to get something I could work on. Then there was the guy who fell in love with C++ templates, and thought they were the solution to every problem. The stuff was almost as unmaintainable as much academic based open source code. We called it "write-only code."

Although, working there with people with Ph.D.s in cryptography or on the C++ standards committee taught me alot about the quirkiness of low level, bit-manipulating code. Things I've long since forgotten, other than I remember that true pseudo-randomness requires a lot of work, and why to never, ever trust any code that uses C bit-fields.

MathAndCode · Post by **MathAndCode** » October 2nd, 2020, 11:01 pm

hkoenig wrote: ↑
October 2nd, 2020, 9:50 pm
Starting in the last century I did contracting at Microsoft, off and on, for about ten years, converting low level Windows code (registry management, video codecs, DRM, Silverlight, etc.) to run in the MacOS environment. Nobody wrote understandable or debuggable code, as far as I could tell. I think they were penalized for every comment or any extra whitespace character or any variable name longer than three characters, and lost stock options for producing actual documentation or design specs ("the source is the documentation" I was told once).

Was the purpose of this to prevent others from being able to steal the code or simply because of a misguided emphasis on "efficiency"?
Also, I sometimes attempt to get rid of unnecessary lines when coding, although I am nowhere near as bad as Microsoft apparently was (and possibly still is).

blah · Post by **blah** » October 3rd, 2020, 7:50 am

hkoenig wrote: ↑
October 2nd, 2020, 9:50 pm
why to never, ever trust any code that uses C bit-fields.

... Why do you never trust code that uses bitfields? I don't think I've ever heard that before.

hkoenig · Post by **hkoenig** » October 3rd, 2020, 10:47 am

Bitfields-- if I remember correctly, they were never defined properly (at least as of 2005 or so), so they were compiler implementation dependent. The standard didn't define where the empty space would be if the bitfields didn't conform to byte or word boundaries. (You have a 3 bit field, and a two bit field. Where are the other three bits in the byte? Before, between, after or something else? And what about three, 3-bit fields? Are they byte aligned or word aligned? (And all that assumes 8 bit bytes, which long ago was not assured, either...)) Endianness came into play, made worse by the fact that Unix and C were developed originally on the PDP-11, which was neither big nor little endian. Bytes of long words on the PDP-11 were stored 3412 there, where littleendian intel is 1234 and bigendian 68xxx/PowerPC 4321. (Working from decades old memory here, so could be mis-remembering.) Also, the endianness within the bitfield itself wasn't required to match the device endianness. The Unix developers "punted" the whole problem, and it never got fixed.

In applications using bit level manipulation and storage, like codecs and cryptography, endianness matters, and you have to be especially careful to read/write out the bits in the right order. Off by a single bit, and your music video plays back as garbage. If you use bitfields with unions and structs and such, your code can change underneath you because your compiler got an update and the implementors changed the way bit fields are stored without realizing it. The compiler passed all the extensive validation suite tests because the validation suite only tests the defined standards. So you break and have no idea why. So all the low level code I worked with went to the trouble of defining all its bitfield type manipulation operators to control the placement of data, even though you did nothing wrong. That was one of the first places I had to get compatibility to work correctly. (Learned a lot of the background from someone who, for a while, maintained a major USB chipset manufacturer's 8-bit C compiler, where every extra bit of memory had to be accounted for because those bits over hundreds of millions of devices adds to the manufacturing cost.)

At Microsoft, projects like the ones I worked on had an "SCM (Software Configuration Management) build" machine where the official code was built. One of my tasks was to set it up so that someone else could actually build the final product. Once the build system was set up for distribution builds, you did not do system OS or compiler updates, to make sure the process was completely reproducible. Once the "golden master" was distributed, that machine got stored away, and a new machine set up for the next version. The idea was that if necessary, it was possible to completely reproduce the software sent out to customers, I never found out if that ever actually happened, or was even useful.

Sorry to get so far off topic, but that brought up all sorts of memories of things I no longer have to worry about...

mniemiec · Post by **mniemiec** » October 3rd, 2020, 3:24 pm

hkoenig wrote: ↑
October 3rd, 2020, 10:47 am
Bitfields...

The problem wasn't bit fields; it was unions. Using bit fields alone should have worked fine, as a means of packing data in an abstract way, although I've almost never used them, nor felt the need to do so.

The language didn't specify how padding worked in general, even without bit-fields. union { long l; char c[4]; } would give different behaviors on 80x86, 68000, and PDP-11. It was even worse if structs were involved, because some machine architectures (like PDP-11) required word-aligned objects to be located on word boundaries, and others (like 80x86) liked to do that anyway for CPU efficiency, even if it was not required, so compilers often introduced hidden padding bytes that would not easily be noticed by code. It didn't help that Microsoft short-sightedly designed its external file formats without alignment in mind, so when files were read into RAM, structure elements would be on unaligned boundaries. Code had to be loaded with non-portable pragmas to ensure that just the right amount of padding (usually none) was included.

Also, union { double d; char c[8]; } can't even pretend to be portable.

blah · Post by **blah** » October 3rd, 2020, 4:02 pm

hkoenig wrote: ↑
October 3rd, 2020, 10:47 am
Bytes of long words on the PDP-11 were stored 3412 there...

2143, actually. I just tested this on an emulator: