What follows is a list of things I wish the Z9 had / could do better. I believe these are actually viable – I’m avoiding the common but perhaps unrealistic items like massively improve dynamic range or noise performance or whatever.

Autofocus

Better low-light autofocus

The Z9’s not bad, but it could be better – all cameras could – and in particular I’d love to see some of the caveats eliminated (like having to compromise between accurate exposure previews and autofocus performance).

Better red light autofocus

Purportedly (per chatter on the interwebs) mirrorless cameras typically only use green and/or blue sensels for autofocus, not red. I’m not sure how accurate that is – it’s a strange choice on the face of it, and at least partly false since you can focus on a purely red object – but it does partially track with the actual behaviour of the Z9 (and the Z7 before it), which is to really struggle to autofocus under predominately-red light or with purely red subjects.

This is particularly a problem underwater, and of course in conjunction with many low-light focus aids such as on some Speedlights and strobes.

For my typical subjects and subject matter it’s not a big deal, although in a way that just makes it even more prominent when I am in that situation.

Better subject recognition

This is a broad area, but any improvement in any direction would be good. Things like:

• Recognition of a wider range of subjects (particularly wildlife).
• More reliable detection of eyes (as opposed to e.g. ears & nostrils).
• “Iris” detection or whatever you want to call it – the ability to focus specifically on the iris rather than e.g. eyelashes.
• Better recognition of subject’s heads when they’re not closer to the camera than any other part of the subject.
  
  All too often the animal is in a profile view, or even facing away from me but with their head / face / eyes still in view, and the Z9 very often loses the face and reverts to “centre of mass”, which is usually the animal’s side, or butt. The Z9 really needs to fixate on the head / face / eyes if those are anywhere in view, irrespective of their position relative to the body.

EVF eye tracking

I haven’t used the Canon R3 – and the reviews of its eye tracking are mixed, indicating it’s not quite there yet technically – but it’s clearly going in the right direction with its EVF eye tracking. This is clearly the superior way of selecting your subject / focus point placement.

Less fixation on detected subjects in 3D Tracking

If there’s a subject detected anywhere in the frame, the Z9 will always focus on it in 3D tracking mode, no matter where the focus point is. This is incredibly frustrating and hostile behaviour, especially while subject detection has so many false positives and doesn’t reliably prioritise the right part of the subject (e.g. ignoring the head in favour of the butt).

Instead, it should lock onto the detected subject only if I actually put the focus point over the subject detection box and then engage it. Otherwise, it should ignore detected subjects and focus on what I told it to.

It’s permissible if there’s leeway here, to allow for imperfect positioning of the focus point vs the subject, such as with rapidly-moving subjects. This could be something that’s configurable, to suit people’s differing tastes and needs for how “generous” the camera should be regarding precise placement.

No trade-off with correct exposure preview

All Nikon Z cameras to date – perhaps all mirrorless cameras? – force an unfortunate trade-off between autofocus performance and accurate exposure previews. I believe this is largely a false dichotomy.

The autofocus sensels are on the image sensor (as opposed to a completely separate sensor as most DSLRs) and their gain setting (ISO) seems to be tied to a sensor-wide value. Their performance relies on having a strong signal (i.e. enough light). Thus it’s important that the gain be as high as possible (without clipping). But that might not be what you want for the final exposure – perhaps you’re trying to preserve brighter tones elsewhere in the frame, for example. Thus your autofocus system might not be getting as much light as it’d like, and it performs poorly as a consequence.

The Z9 allows you to either see an accurate exposure preview – at the expense of poorer AF performance if your subject isn’t very bright – or inaccurate exposure (similar to the optical viewfinder experience).

I believe it could do the best of both at only minor inconvenience to dynamic range accuracy – it can adjust the sensor’s ISO to suit the autofocus system, then digitally scale the exposure in the EVF to represent your exposure settings. This does potentially mean crushing the blacks or blowing the highlights in the EVF’s preview (no such issues with the actual photos) but that’s a minor inconvenience in comparison to the alternatives.

Making the ‘strength’ of this tuneable could also help suit every individual’s preferences (e.g. allow up to N stops of such internal adjustment).

Note that it could also in theory adjust the autofocus sensels independently to the imaging sensels used for the EVF / LCD image, and that would of course be the optimal solution. I’m just not sure how viable that is for technical reasons. I also suspect that as autofocus systems continue to evolve into scene- and subject-analysis systems, they’ll need essentially the entire image anyway.

Same autofocus in video mode as stills

This applies broadly – right now in video mode you have more limited options (e.g. no 3D tracking, only the less reliable “subject tracking”), you can’t use custom buttons at all for customised autofocus engagement, and you also have a way less performant autofocus system in general.

It’s baffling that there are these differences. The limitations on button configuration are just arbitrary. And I don’t know what camera resources they’re overloading between autofocus function & video recording, that preclude them both being used simultaneously, but they should stop it. Add more dedicated hardware. Do whatever it takes to make autofocus work identically whether you’re doing stills or video.

It’s clear Nikon pushed harder than ever to make the Z9 a good video camera, so it’s baffling why they didn’t address these flaws along with the boost to recording resolutions, bitrates, and formats.

To elaborate, autofocus in video mode on the Z9 is disappointing. It doesn’t work correctly a lot of the time – outright refusing to focus, or focusing stubbornly on the background no matter what you or your subject do, or just simply missing acceptable focus. Switch to stills mode and autofocus often works perfectly, in comparison. In fact it’s such a dramatic disparity that I sometimes switch to stills mode temporarily just to autofocus. Yes, it’s very frustrating and I miss critical moments, but the alternative is all-too-often that I can’t get anything in focus at all.

Manual focus should of course not be the ‘workaround’, but even aside from the principle of that, it’s just not possible to accurately manually focus while recording video when you have 8k video (~33 megapixels) in a 1.2-megapixel viewfinder. Even in 4k (~8 megapixels) it’s very challenging. Let-alone whether you’re skilled enough to track a moving subject anyway.

Camera modes

Motion-aware aperture priority

The camera should be able to set the shutter speed automatically based on actual subject & camera movement. e.g. if I’m photographing a bird that’s perched, essentially immobile, in limited light, the camera should automatically drop the shutter speed in order to lower the ISO and thus minimise noise. If the bird suddenly starts moving, it should instantly raise the shutter speed to whatever is necessary to freeze the bird’s motion.

In all of this it should understand what shutter speeds are viable given the degree of perceived movement involved – factoring in focal length and recent image stabilisation performance – and including the recent history of camera movement so that it adapts to different users and situations (e.g. buffeting winds, being on a moving platform, etc).

Some cameras – like GoPros – already do a limited variant of this whereby they end an exposure early when they detect significant camera movement. Especially in video mode where you can benefit from inter-frame noise reduction, this is what helps make GoPro footage look exceptionally-well stabilised while remaining surprisingly consistent in exposure and noise levels.

The degree of ‘freezing’ could be configurable along two dimensions:

• Strength. Different folks have different tolerances for blur, so being able to trade-off between pixel-perfect sharpness and noise is important.
• Subject-only vs whole scene. Maybe you want to freeze your subject but don’t care about the background, such as when panning for a bird in flight or moving vehicle. I expect this’d be what most people want most of the time. But sometimes you might really want to freeze the entire scene, even if you’re panning.
  
  This is analogous to exposure compensation settings for use with flash.

Subject-aware shutter priority

I’m quite surprised we don’t already have this, on at least one camera somewhere.

I want the camera to adjust the aperture intelligently to account for the subject’s depth and focal distance. So that I can just set it to basically e.g. “whole head in focus”, and not worry about micro-managing the settings as the subject moves closer or further away.

It should handle multiple subjects too – e.g. for a group photo where people aren’t all neatly in the focus plane it should adjust the aperture to compensate.

Whether intrinsically or through e.g. lens profiles, it should account for curvature of field.

This could be flexible like Programmed Auto mode, where you could use a dial to adjust the depth of field if the camera’s selection doesn’t precisely suit your preferences (since you’ll be making trade-offs between in-focus subjects and background blur).

Controls

Automatic grip selection

I wish the camera could automatically detect which grip I’m using, so that I don’t need to micromanage it with a lock control.

Possibly this could be implemented through some kind of contact detection in the two grips, to tell which is being held? I know it can’t use camera orientation, since it’s not uncommon to use either of the grips when they’re not oriented vertically.

It of course needs to be very reliable (erring, if necessary, on the side of allowing use of the controls vs ignoring them), and work in a wide variety of situations. e.g. with or without gloves, whether the camera / hands are dry or wet, across a wide temperature range, with hands of various sizes, with hand-holds of various types, etc.

Delete & undo

Currently to delete you have to push the delete button twice, because it prompts you to make sure you want to perform the delete. This is nominally required because deletes are immediate and permanent.

The vast majority of the time, I do want to perform the delete. Very rarely is it a mistaken button press.

Doubling the button-presses required gets real old when you’re deleting thousands of photos (and while it’s faster to delete them on a computer, I prefer to do an initial cull in-camera to avoid wasting space on my computer and backups – plus if I’m travelling I may have limited card space and cannot wait until I’m back home).

It also doesn’t add much actual safety – it’s just hard-wired into my muscle memory to double-tap delete, and occasionally I’ll delete something I actually didn’t want to, as a result. So the current system is inefficient and doesn’t work as intended.

What it should instead do is follow user interface best practices dating back to the eighties (if not earlier) – make the delete operation undoable, and therefore not need confirmation every time.

This could be implemented in a variety of ways, each with slight differences in trade-offs. Even a rudimentary implementation, that only allows the most recent delete to be undone, would still be a huge improvement.

An even more robust system would likely not be much more work – e.g. move deleted photos to a separate ‘bin’ folder, just like on a computer. The camera could also make them auto-purge, so if the card is full it’ll start permanently deleting files from the bin as needed to recover space.

Consequently it’d be much safer – even against completely accidental delete button presses – and in-camera image review would involve about a third fewer button presses (currently two deletes plus left or right to move between images for comparison).

Note: how you perform the undo, I’m not sure about. The most common case would be undoing the most recent delete so there should be a way to do that which doesn’t completely interrupt your image review (i.e. no making you use the Menu button or otherwise switch away from the image you’re currently looking at). It could be simply by hitting the ‘i’ button and having an ‘Undo’ option in that menu.

Fix the portrait grip lock switch direction

It currently rotates opposite to the main power switch (on the landscape grip), which is weird and confusing. i.e. push the tab away from you to unlock the portrait controls, which on the landscape control turns the camera off. When I pick the camera up I should be able to use the exact same motion to enable the controls irrespective of which grip I’m holding.

I’d love something that goes even further and lets you actually turn the camera on from the portrait grip controls, but I don’t see a good way to do that (it would interfere with the function of selecting which grip you want to be active). Though this would be moot if the aforementioned automatic grip selection were supported.

Subject detection configuration via customised buttons

It’s great that the Z9 returns the functionality that the D500 et al had years ago, of letting you assign AF-ON plus a specific focus area mode to many buttons. This is super essential for any camera in many circumstances – especially wildlife where you’re often dealing with obscured or unusual subjects. It was particularly remiss of Nikon to leave this out of all their prior Z-mount cameras, since they had such subpar autofocus systems.

However, it still has some limitations in terms of configurability. e.g. you can configure a button to turn subject detection on or off, but it has to be independent of actually engaging autofocus. And you can’t configure it to change the subject detection mode (e.g. from ‘All’ to ‘Animals’).

Ergonomics

Lighter

I almost didn’t call this out, except Canon proved with the R3 that you can shave a significant amount of weight with seemingly no downside. That would be appreciated – it’d be right in line with Nikon’s impressive improvements to their telephoto lenses to make them much lighter than their DSLR forebearers.

Symmetric function buttons in portrait vs landscape grips

It’s baffling to me that there’s three customisable buttons next to the lens mount for the landscape grip, but none for the portrait grip; you can only reach one of the three buttons in portrait mode.

They should add another two buttons for the portrait grip, matching the relative positions of the landscape mode.

There’s still a challenge of button function, if they continue to share a button between the grips, since in landscape mode it’s under your pinky or ring finger while in portrait mode it’s under your index or pointer finger. Ideally the camera would switch automatically depending on which grip you’re actually using, iff there’s a reliable way for it to detect that. If not, it might be worth adjusting the button placements so that you have completely independent button sets between the two orientations (and at least mirror the settings between each set – though I wouldn’t object if they could also be customised independently).

Smaller

It could be smaller without compromising ergonomics – maybe 10-20%. At least w.r.t. the grips. It barely makes the list, though, since the main way to make it substantially smaller is to remove the portrait grip, which arguably defeats the point of a top-line camera. That said, the Z8 (and the Sony Alpha 1 before it) have shown that there is a strong market for a flagship without built-in portrait grip.

Before I got the Z9 I was pretty sure a built-in portrait grip was not for me, though after getting used to the Z9 I’m now more on the fence. I’ve had detachable portrait grips for prior cameras, and I recognise that they just don’t feel as good as a built-in grip. They’re also heavier, and less robust.

EVF / LCD

Larger LCD

I don’t know how it might work ergonomically – good placement of physical buttons is definitely the priority, and there’s only so much space available on a reasonably-sized camera – but it would be really nice if the LCD were substantially bigger. Compared to what we’re used to today with phones, camera LCDs are tiny.

It would need higher resolution to compensate. I’m not thrilled with the Z9’s LCD pixel density, but it’s okay. As long as the pixel density didn’t decrease, it’d be okay.

Lower latency

Though the Z9’s EVF latency appears to the best of any mirrorless camera to date (according to various test reports I’ve seen), there is still visible lag (even in 120Hz mode). It’s not a big deal by any stretch, and the vast majority of the time I don’t perceive it. It’s only if I’m moving really rapidly, especially if changing direction frequently. However, even if I don’t typically perceive it, I wonder if it’s nonetheless having a negative impact on my performance with the camera.

I doubt that higher refresh rates are the solution, at least not directly. The problem is the time it takes for photons hitting the sensor to be reflected in the EVF. It might be technologically impossible to reduce the delay entirely (even before you hit the physical limits), but I hope there’s still improvement possible.

Higher resolution EVF

This didn’t initially make my list, but after much use I do think the Z9 EVF is a tad soft. I can see the pixels, and I do find it’s a bit tricky to judge focus precisely (without digitally zooming in) – moreso than with an optical viewfinder.

Possibly related, I’m a bit mystified as to why image review in the EVF seems so blocky and pixelated compared to on the rear LCD, given the latter is objectively much lower resolution. It seemingly can’t be a hardware problem – perhaps a software error? Whatever it is, fixing it would essentially increase the resolution too, for image review.

Note also that I’m focused on the EVF specifically here. Curiously I don’t see the pixels on the LCD, or at least I never notice them. I think because the viewing distance is so much farther away. I certainly wouldn’t object to a higher pixel-density LCD too, but it’s not something I really need.

The rest & the usuals

None of these last few items are what I would call critical nor actually highlight. They tend to improve incrementally over time in any case. Those improvements are important and appreciated but not noteworthy unless there’s an unusually big leap.

Though admittedly it would be particularly good to at least match the state of the art w.r.t. image quality (or even of much older cameras like the D850).

• Less noise.
• Higher resolution. Though I don’t want to sacrifice anything for minor resolution gains – e.g. to go up to 60MP. For a major jump – e.g. to 100MP – I might be willing to trade off other aspects of performance.
• Better battery life when the camera is left on. As much as its start-up delay is relatively brief compared to most cameras, it’s still far from zero and in any case it costs time to locate & operate the power button every time I bring the camera to my eye.
• CFExpress 4.0 support, for at least a doubling in write speed (although the Z9 currently uses barely more than half the available write performance of CFExpress 2.0 anyway, so in fact there’s room for nearly a 4x improvement with current technology).

I started out as a one-person shop, doing my own things, selling shareware. I had no manager nor technical lead. I had to make all my own decisions, in all aspects, without guidance or assistance.

Subsequently, during my four years at Apple, I did have a manager, but they focused on people, not the technical – myself and/or my colleagues collectively made the technical decisions, and provided technical leadership, and effectively set the product direction. My managers were there to make that as easy as possible for us.

Over my nearly eight years at Google, I observed the tail half of a major cultural transition for Google. Long before I started, Google had explicitly laid down a culture where managers were not product / technical leads. The two roles were physically separated, between different people, and they operated independently. Managers focused on people – career growth, happiness, basic productivity, & skills – while tech leads focused on the technical, the product. In fact the manager role was so principled about focus on people that managers would sometimes help their direct reports leave the company, if that was simply what was best for those people for their own success & growth. And, to be clear, not in a “you aren’t working out” sense, but for engineers that were excellent and simply didn’t have deserved opportunities available to them at Google.

By the time I joined, that culture was half-gone, but still present enough in my division for me to experience it. But by the time I left the culture was heavily weighted towards managers being technical leads.

In my nearly three years now at LinkedIn, I’ve completed that arc. LinkedIn culturally & executively emphasises managers as technical / product leads even moreso than Google ever did. As far as I’ve been told, LinkedIn always has (meaning, this is presumably the culture Yahoo had too, from which LinkedIn forked).

Having experienced most of this spectrum, I finally feel qualified to pass judgement on it.

I immediately, intuitively recognised & appreciated this at Google, but now I’m certain of it.

People management & (technical) product leadership are fundamentally at odds with each other. The needs of individuals are often at odds with the needs of the product. The product might need Natalie to really focus on churning through a bunch of menial tasks, but to evolve, Natalie might really need design experience & leadership opportunities.

Having one person (in authority) try to wear both hats creates conflict, bias, and inefficiency. It discourages dialogue, because you can never really trust where the polymorph stands. The roles require different skillsets, which rarely coexist in a single person and in any case are difficult to keep up to date in parallel. Context-switching between them is burdensome. It creates a power imbalance and perverse incentives.

Even if an individual is exceptionally talented at mitigating those problems, they simply don’t have the time to do both well. Being a product or technical lead is at least a full-time job. Likewise, helping a team of any real size grow as individuals requires way more hands-on, one-on-one attention than most people realise. It’s hard enough being good at either one of them alone – anyone that attempts doing both simultaneously ends up doing neither effectively.

I’ve had the opportunity to be both a technical lead only and a manager only. This is quite rare in the tech industry. I deeply appreciated being able to focus on just one of those roles at a time. I could be consistent, deliberate, and honest. I could, as a manager, tell people exactly what I thought they should or shouldn’t work on, irrespective of what the product(s) need, because I knew the technical lead(s) would worry about those angles. Conversely, when I was a technical lead, I could lay out what was simply, objectively best for the project, uncomplicated by individuals’ interests. In either case, there was real, other human being that could be debated with, as necessary, to find happy mediums.

Yet beyond just being more efficient and effective, the serendipitous consequence was that it gave agency to the individuals – whenever a conflict arose between people and products, it was revealed to them, and the implicit decision about it at least in part theirs to make. Most importantly, they knew that whichever way they leaned they had someone in their corner who had their back.

(Of course, sometimes they didn’t like having to make that decision, but putting it on them forced them to take control and responsibility for themselves, and evolve into more confident, happy, motivated developers.)

I suppose it’s no surprise that companies tends this way – to conflate people with products. These days, for many big tech companies, people literally are the products, and their humanity inevitably stripped away in the process. People are “promoted” into management from technical positions, and often by way of the Peter Principle, are not actually good people managers, nor able to relinquish their former role and ways of thinking. A hierarchy of technical leads in manager’s clothing becomes self-sustaining, self-selecting, and self-enforcing.

The question is: what’s the antidote?

Acknowledgement: I was inspired to pen this post by reading Tanner Wortham‘s Why Manager as Product Owner Will Usually Fail, which is essentially positing the same thing albeit in different terminology.

I was surprised by how well he spoke. I can’t recall a better speech, at least not from recent memory. I suppose it’s not exceptional, given he was of course summarising his most recent book, and as someone who’s lived and breathed the topic for the last decade, he very well should be well versed in it.

He didn’t say anything I didn’t already know, but rather everything I didn’t know I knew. (sorry, couldn’t resist) I’d call him brilliant if he hadn’t so eloquently convinced me that his opinion is just so blindingly obvious – that copyright today, the legal interpretation of ancient laws – simply hasn’t been applied correctly to the world of today (the “digital” world, I suppose). He was swift to clarify that he was not an abolitionist, when it comes to copyright, but simply wants to see it applied correctly. It seems hard to argue against that; he’s clearly right, and you’d have to wonder why anyone would disagree.

Though of course he pointed out why they would – selfishness, greed; money, and power. Which apparently has segued him into his focus for the next decade, once he takes up residence at Harvard next year, on political corruption. I particularly resonated with the way he presented the issue, which wasn’t alarmist or extremist, but rather in a kind of forlorn way. Everyone is deservedly cynical about politics and the massive corruption perceived within it, so again it seems like he’s just comfortable and certain that everyone already knows this, and he’s not trying to condescend his audience by stating the obvious, but rather encourage action to effect real change.

I bought his book on the way out, leery as I was of the potential commercialism or hypocrisy in doing so – it’s all about selling books as a public speaker, isn’t it? :) – so if nothing else that’ll keep me occupied through a few otherwise idle evenings. I’m not sure what it’ll add, given his presentation seemed to convey his points so well already, but hopefully it’s just as impressive.

Worth playing with, anyway.

In reality, I don’t decide “hey, I’m going to learn X% of mechanics, because”… I say “crap, my car’s broken again… time to get out some books on introductory mechanics”. It’s the problem that drives us to learn, not some arbitrary god figure clicking insistently on the bookshelf.

So I’d like to see Sims-style games take a different approach in future – make it part of the gameplay that you have to challenge your Sims, which forces them to learn, to adapt, to evolve. Perhaps the way-hyped “Spore” will do just that… who knows.

It’d certainly be much more interesting to think up carrots with which to improve the minds and bodies of my Sims, than to just keep applying the stick all the time.

But another reason perhaps is that the arguments in favour of microkernels are largely fluff. The advantages in stability and security oft-touted don’t go nearly as far as the proponents would like us to believe. The common line is:

“For example, each device driver runs as a separate user-mode process so a bug in a driver (by far the biggest source of bugs in any operating system), cannot bring down the entire OS. In fact, most of the time when a driver crashes it is automatically replaced without requiring any user intervention, without requiring rebooting, and without affecting running programs. These features, the tiny amount of kernel code, and other aspects greatly enhance system reliability.”

The MINIX 3 website

Now, let’s just hold our horses there. Sure, if an unused driver crashes, it can be reloaded and none will be the wiser. But that’s not how it works, is it? See, drivers have state. And when they crash, they lose that state. All the hyperbole in the world isn’t going to magically restore it. Even if you could, should you? Having the reloaded driver restored to the state it was in just before it crashed may not do any better the second time around – it may just crash exactly the same way again.

And what about everything else that’s using the dead driver? Well shit. I mean, we just don’t think about these issues properly. If I’m writing to disk via some file system driver, which crashes sometime during the write, I’m in trouble. Can the OS automagically restore the driver and continue or repeat the write without me knowing? I doubt it. So what happens? Well, I guess my program will get an error back from the write. But what if the write did actually go down to the physical media before the crash? Oh oh.

So we have all this journalling and so forth… but really, it’s a bad solution in the long term; journalling just adds more places where things can go wrong.

So now our driver has to be able to figure out what it’s already done. Maybe it can do that, sure. But does it? In today’s drivers? I doubt it.

You see, the focus on microkernel’s is really just taking to a wheat harvest with a pocket knife. It’s not thinking on the appropriate scale. What microkernels critically provide is simple, defined and protected interfaces between modules. That’s all it is.

But, you see, the best place to do everything is at compile time, not run time. Errors at run time piss off your users, since they’re the ones running them. No, what we need are smarter compilers. Compilers with defined limits on parameters, more explicit type checking.

And yet I’m a big fan of Objective-C? Why is that? Can these two bipolar titans be married? I like to think so. See, let’s take to an example to explain this simply. I have a function for doing logs, like so (in traditional C):

double loge(double x) {
  /* Perform magic arithmetic here */
  return result;
}

Now that’s not much good, really. What if someone tries to pass a negative value, or zero? Well that’s not much good; natural log’s not defined (in the real domain; we are working with doubles here) for negative values (and loge(0) goes to negative infinity as a limit; it has no actual value). So, sure, the standard way of going about this is to either use algorithms which handle this implicitly, or more generically to do parameter checking:

double loge(double x) {
  if (0.0 >= x) {
    // Barf!!!
  } else {
    /* Perform magic arithmetic here */
    return result;
  }
}

Now, here’s the problem… how do we “barf”? Do we return a symbolic value that represents NaN (Not a Number)? Perhaps we just return 0? Perhaps we raise an exception (if we’re using C++)? Perhaps we call exit()? Perhaps we just while (1) {} just to annoy the caller?

Is the compiler smart enough to realise that we’re using a bad value when we invoke the function? Nope. Even if I use assert() or some similar standard procedure, the compiler won’t even issue a warning on the parameter. Compilers just don’t do range or domain checking (at least, gcc doesn’t). It would be a good start if they did.

So what we need are stricter types – we need definition’s of domains and conditions. Imagine, if you can, if we could do this:

double loge(double x) where (0.0 < x) {
  /* Perform magic arithmetic here */
  return result;
}

In this mythical language – let’s call it ++C – the compiler would issue an error if we tried to violate the explicit condition we’ve provided. Thus it would be impossible, given a suitably intelligent compiler, to call this function with an inappropriate parameter.

Of course, people will say “but how is the compiler to know what values we’re going to use if we’re taking them from the user, for example?”. “Output one, accept any”. Sources such as files could have any value of course, so the compiler should assume any possible value. Thus, if we wanted to take our value “input” and pass it to loge, we’d have to do our own explicit range checking in the caller.

So, we’ve just shifted the problem to a different place, right? Well, yes and no. Basically yes, which is a good thing on it’s own. Now the caller – which is going to know more about the data than the callee – must make the decisions about how to handle invalid data. Excellent. We move input validation right to the very top of our program, where it should be. This is the way it should always be done anyway – it keeps your core code simpler, leaner and faster, since it doesn’t have to perform redundant parameter checking. In reality, because there is no compiler enforcement of parameter limitations like this, we end up with a lot of redundancy as checks are put in at multiple layers, just to be safe. Even then, we still miss things.

This could even be integrated into existing compilers in a completely backwards compatible way, by having some kind of error marker which the compiler could use to determine which code branches should never intentionally be taken. For example, the compiler could see the following:

double loge(double x) {
  if (0.0 >= x) {
    INVALID_PARAMETER(x);
    // Barf!!!
  } else {
    /* Perform magic arithmetic here */
    return result;
  }
}

It could perform the same kind of analysis as talked about previously, but instead of an explicit addition to the language syntax, it just looks for this INVALID_PARAMETER macro invocation. If it’s analysis indicates it’s possible to reach this point in the code, it can issue an error – or at the very least a warning. This can be tied in with dead code stripping as well; if the compiler can prove at compile time that no INVALID_PARAMETER‘s are reached, it can remove all the code in the same scope as the invocation. Fantastic!

I should add, there’s a language called D which I believe tries to adopt something like this. I’m yet to encounter a D compiler – although I haven’t looked – but until it makes it into gcc, it’s not going to get the large-scale support such features need.

So, with this advanced compile-time analysis, we get the benefits of microkernels – and more – without the runtime performance costs. We can all follow the Linux standards and code like psychotic schizophrenics, and still get the safety we need.

Well, okay, so there’s a lot of other stability and security issues beyond just this, but I think it alleviates a major fraction of the problems. Throw in more intelligent compiler behaviour regarding pointers, arrays, etc, and we’ll be set.

[Incidentally, I believe it was bzip2 that was the top performer. sitx did reasonably well, but I think has more overhead than zip or gzip formats, and consequently lost out to them on a piddly ~70k archive.]

And I was just reading this article here, which talks about a company called Neuros Audio soliciting feedback from the “hacker” community as to what direction they should take with their future product(s). They go on about how important it is to them to get involved with the hacker community, and how they’re using Linux 2.6 and uber-leet audiophile-worthy DAC/ADCs, etc… and then at the end there’s the note:

More details, including a downloadable 18-page Word document describing the current development board specification, can be found here.

Ummm… what? It’s a Word document… for a document supposedly aimed at Linux users. Duh.

So I thought… really, there’s always going to be someone complaining about the format, just as I found out myself with Rotated Windows. So why not provide a whole bunch of different formats? The immediate pessimistic response is because it makes the download process more convoluted – asking users to pick a format, when they may not really be aware of the differences between them or what their computer can work with.

But the optimist would say we just need better technology, that allows us to indicate which formats we prefer, and have the conversion performed – on the fly if possible – at the server end. If the conversion is done there (rather than using importers or conversion tools locally, which is difficult for many average users) it can be done properly – i.e. they can proof each differently formatted version to ensure the conversion went smoothly.

I call this very trademarkable idea Chameleon. In my mind the browser would maintain a list of preferred document formats for different MIME types, which it could submit to the server when requesting the files (or when otherwise prompted, if a 2-stage fetch is necessary). The server could then do what it could to work within that preference…

…I think it’s a sweet idea. A trivial demo would be to use php and ImageMagick to convert images on the fly to whatever format each user has previously stated they prefer. Beyond that you could use converters for different text formats, and then perhaps even more complex things like spreadsheets, databases, etc.

So if anyone makes their millions of this idea, don’t forget about me. ;)

Of course, that’s an annoying way to program, and accident prone. A much better way would be if the compiler could do this magically for you, by pulling the struct apart behind the scenes and storing it as such. In this way cache would be used much more effectively, memory bandwidth usage reduced substantially, and everyone made happier.

There are of course issues with this, because modifying the length of the array would be more expensive with multiple arrays behind the scenes, and it may be difficult to determine automagically which approach is better [in programs which have multiple distinct access patterns to that array]. Still, it seems like it should be an option – there’s plenty of cases where such an optimisation could be made, and yield significant improvements.

Now if only I knew the intimate ins and outs of the gcc 4.x source. D’oh. :)

The problem I had was figuring out how to get notifications of every file notification. It can be done – Spotlight obviously gets such notifications. But it can’t be done using kqueues (as far as I can tell, anyway), and I’m not aware of any other public API for doing so. I’m not really that interested in the idea to reverse engineer half the system trying to discover the private mechanism… so the idea languished.

Until two things struck me – Apple just posted this article on the kern_control.h & kern_event.h headers in /usr/include/sys/…. it seems to me – at a cursory glance – these might provide the mechanism for receiving general I/O events, which could be broken apart to discover their nature, scope and relevant details. Of course, these do seem somewhat orientated towards kext’s or similar kernel-mode code, but there is ioctl hooks that look interesting… definitely something to research.

But the big epiphany was the just realising that I can use the public Spotlight API in various hacky ways to do what I want… for example, list all files modified in the last 60 seconds or so, which will update constantly with all newly modified files… provided Spotlight queues all prospective results, and doesn’t drop any due to time-outs or similar things. Definitely worth investigation.