MishMash

Grand Challenges for Computing Research

The UK Computing Research Committee (UKCRC), with support from EPSRC and the National e-Science Centre (NeSC), is running an exercise to put together a set of “grand challenges” (“ambitious, long-term research initiatives that might benefit from some degree of national and international coordination”) for computer science research.

The current draft proposals include one called Memories for Life [pdf, 28Kb], which is introduced thusly:

People are capturing and storing an ever-increasing amount of information about them-selves, including emails, web browsing histories, digital images, and audio recordings. This tsunami of data presents numerous challenges to computer science, including: how to physically store such digital memories over decades; how to protect privacy, especially when data such as photos may involve more than one person; how to extract useful knowledge from this rich library of information; how to use this knowledge effectively, for example in knowledge-based systems; and how to effectively present memories and knowledge to different kinds of users. The unifying grand challenge is to manage this data, these digital memories, for the benefit of human life and for a lifetime.

This is interesting. It hadn’t occurred to me that the frustration I feel at the absence of day-to-day tools to store, manage, index, associate, and find information (email, documents, notes, pictures, bibliographic references, events — the things I work and play with every day) could be generalised to this degree. Reading this done, it is an obvious step. The proposal notes that, “An argument could be made that this was first proposed by Vannevar Bush in the 1940s

, which reminds me that I still haven’t found the time to read “As We May Think”.

Is it an implication of this vision that the notion of “delete” would be rendered obsolete? Or at the very least, that “delete” would be an operation conducted only for a few very particular reasons? If I could rely on my personal store of data not to be compromised, then there is no reason why I should delete things. Of course legislation like the Regulation of Investigatory Powers Act which provides sweeping powers to law enforcement agencies to demand access to encrypted data more or less at a whim would ensure the continued value of such an operation.

One of the “exemplars” suggested in the proposal for a 10—year time scale is the development of “Newpaper”. I want this last year!

Develop smart electronic paper that lets any-one (even people without formal IT education) write down thoughts, scribblings, draw-ings, or whatever, and have these incorporated into the person s digital memories. In addition to clean sheet information, newpaper also lets people display and annotate existing documents, including their other memories. Newpaper can be used anywhere (bus, bed, bath), so people always have access to their memories.

Although the proposal does acknowledge that, “it may prove … that some tasks cannot be solved without human cognition”, but this brief mention deserves much more prominent display than it gets. Any tools built in this vision must work as prostheses to the mind, not replacements for it.

To have this issue flagged as a grand challenge would no doubt unlock significant funding to address it, though not without risks (copyright and patent issues, for example, are perhaps more likely with ideas generated in this context than in evolutionary improvement of tools by those who need such tools collaborating).

Peter Medawar

“Science is the art of the soluble.”

Links in weblog entry text

I find myself having to restructure sentences, just so that I don’t
end up putting links on consecutive words. Weird.

FT.com article on wireless, Consume Bristol

Ed Mitchell, on the mailing list of the CLAN project, notes an article from last October in the FT.

Written by one John Paul Flintoff, it’s a well written and reasonably balanced article. His take on the similarity of free wireless access and linux suffers from the usual problems of metaphorical comparisons, but has some value nonetheless. Of course, to say that Linux, “has failed to compete successfully with Windows despite being free to use,” is plain wrong — more accurate would be to say that Linux, “has failed to dislodge Microsoft from its effective monopoly of the desktop operating system market.” But there’s no need to get sniffy ;-)

There are some fascinating things crop up from this article about the nature of public mailing lists, as Andy points out. Since he (John Paul) makes an issue of the fact that he isn’t a techie, then perhaps he isn’t familiar with the modus operandi of email discussion lists, and of the fact (is it a fact — apparently not always) that people who post to such lists are fully aware of the fact that what they are saying is open for the world (and not just the list subscribers) to read.

This is quite important in the context of community wireless, too, since if we are interested in enabling the emergence of interesting applications of what is perhaps best characterised as a metropolitan area network, then those uses will surely rely on this open-by-default ethic (that is, you choose to make a communication private) which is embedded in the complex evolutionary family tree of usenet news, email lists (paricularly with web based, indexed archives), web fora, and weblogs. This seems logical to me given that, over an area such as Easton, for example, private interaction is more likely to take place face to face (or by ‘phone, of course) than on a network. Why sit at an IRC terminal when you can meet in the pub? (OK, because I’m at my desk and pretending to work is one answer to that, but you get my point).

On a more mundane level, too, a community project will only gain community support if it gives the impression of being open to the community. And the only sensible way of giving that impression is, er, being open to the community.

Which leads to another very good point made in the article: that this tech is still something of a geek preserve. That really only fully applies to getting nodes set up, though. If you’re within reach of a node, *using* it is no more geek-limited than using a computer with dial up networking, which is patently not limited to geeks, although it equally patently *is* limited by all sorts of factors which collectively make up the notion of a digital divide.

A closing remark: for a journalist to laugh about people discussing how to pitch to journalists is funny in itself. If I was a journalist myself I’d be overjoyed to see people, under my very nose, trying to make my life easier. Like, for example, writing ready to publish articles, or doing my research for me!

Smalltalk for .NET

More .NET news from comp.lang.smalltalk — refactory.com have released
SharpSmalltalk. Smalltalk for .NET. The .NET juggernaut gathers
pace.

Interesting though, I read a discussion recently suggesting that dynamically typed languages couldn’t be implemented fully within the .NET security model. Can’t remember why, even if I could understand it. I wonder if SharpSmalltalk gets round this somehow? It’s too late to investigate.

Generics on the horizon for C# and .NET IL

“alexis rzewski” points out (in comp.lang.smalltalk, bizarrely)
that there is a recent article
on MSDN about upcoming
features for C#. They include full support for generic types,
including a sensible implementation in the common language runtime
such that each specialised version of the generic type doesn’t cause
unnecessary bloat at compile time (as C++ does) or runtime.

I wonder if they’ve been talking to Bertrand Meyer about this? Certainly this should simplify ISE’s Eiffel->.NET compiler somewhat, and also should mean that other languages can make use of generic types written in Eiffel without requiring name mangling.

Use the most powerful language

OK, I’m on a Paul Graham
binge. In Beating the
Averages he discusses the fact that all languages are more or less
powerful than others, but that someone used to a given language can
only recognise that some are less powerful than it, they aren’t
equipped to understand that more powerful languages are so, or how
they are so.

By induction, the only programmers in a position to see all the differences in power between the various languages are those who understand the most powerful one. (This is probably what Eric Raymond meant about Lisp making you a better programmer.) You can’t trust the opinions of the others, because of the Blub paradox: they’re satisfied with whatever language they happen to use, because it dictates the way they think about programs.

I know this from my own experience, as a high school kid writing programs in Basic. That language didn’t even support recursion. It’s hard to imagine writing programs without using recursion, but I didn’t miss it at the time. I thought in Basic. And I was a whiz at it. Master of all I surveyed.

Which is the core of my “what have I been missing” comment in the previous post, specifically so since the feature which Graham notes as present in Lisp and missing in Python, C, Perl, Java, is macros.

And again the impact of languages thing — I’ve a feeling I’ll be quoting Graham a fair bit in the future.

Ordinarily technology changes fast. But programming languages are different: programming languages are not just technology, but what programmers think in. They’re half technology and half religion.

In Revenge of the Nerds, he notes that, “reading [code] at runtime enables programs to communicate using s-expressions, an idea recently reinvented as XML.” I’ve heard this before, somewhere, and still have no idea what it means, because I don’t know what s-expressions are. Once again, I note it here so I can find it again in the future.

So, it seems that the brackety “syntax” of Lisp is in fact the absence of syntax - the program is a representation of the parse tree ordinarily generated by a compiler from syntax-ful code. This seems a bit like using an RPN calculator, where you have to have a mental image of the stack on which you are operating before you can understand what is going on at all, but once you have that image, you can do things more quickly and easily and with fewer key strokes than with a “normal” calculator. It so happens that exposing the “lower level” — the stack — also exposes more power.

Greenspun’s Tenth Rule (what are 0/1 — 9?):

Any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp.

The idea here is that there are certain problems which can only be solved (in these languages) by for example writing an interpreter for a higher level language. Graham links this to design patterns, too:

This practice is not only common, but institutionalized. For example, in the OO world you hear a good deal about “patterns”. I wonder if these patterns are not sometimes evidence of case (c), the human compiler, at work. When I see patterns in my programs, I consider it a sign of trouble. The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I’m using abstractions that aren’t powerful enough— often that I’m generating by hand the expansions of some macro that I need to write.

I have a suspicion that this is going to bite me in future work with modelling tools. What I am trying to do — what I have explicitly said already — is to provide model developers with a more appropriate tool (language, notation) to work in than a programming language. I suspect that this isn’t quite right. Perhaps what I mean is that I want to allow the model developer to use a more appropriate language, which allows them to concentrate on the task in hand and not generate reams of endlessly repeated code. That repeated code might, as it is now, be to handle data, generate graphical displays, and so on, or it might, with a good framework, be the glue code to make the modelling element under development into a plugin for the framework. Either way, repeated code, like patterns, shouldn’t exist.

What does this mean — I need to move to Lisp? ;-)

Abelson and Sussman: “programs must be written for people to read, and only incidentally for machines to execute”.

Hundred Year Language

I find, via Sam Ruby, a speech given by Paul Graham at PyCon.

[There is a wealth of fascinating looking material on Graham’s web site.]

First off, Julia Lerman makes some meta-comments about this kind of discussion, which it would serve me very well to take note of.

I know it is not important for someone like me to worry my head TOOOO much over this stuff, but look what already has happened. I have stopped working to think about this. That is the dangerous part. This stuff is really fascinating and intriguing. But unfortunately nobody is paying me to sit around and think about it!

My brain seems to be wired in such a way that I find it impossible to put these things down when they crop up. So to the article itself. There follow some excerpts and comments. These are included largely so that when I want to find out where an idea came from, I can search my weblog and find this posting.

I think that, like species, languages will form evolutionary trees, with dead-ends branching off all over. We can see this happening already. Cobol, for all its sometime popularity, does not seem to have any intellectual descendants. It is an evolutionary dead-end— a Neanderthal language.

…

At any given time, you’re probably happiest on the main branches of an evolutionary tree. Even when there were still plenty of Neanderthals, it must have sucked to be one. The Cro-Magnons would have been constantly coming over and beating you up and stealing your food.

This raises interesting questions about how new languages become, and remain, mainstream. Python, for example, has been steadily gaining traction. Perl has enormous popularity. Smalltalk and Eiffel have small but dedicated followings, and continue to resist total eclipse. Paul Graham himself is a Lisp man. I realise that these are all languages, and might not be regarded as language branches, but every language is in some sense a branch.

Graham also points out that language evolution is different from species evolution because branches can converge. This is presumably because programming languages are memetic, not genetic, constructs. Their evolution rides on the mental ability of humans, and is not a process of random mutation and selection by external pressure. More specifically he suggests that the FORTRAN branch is merging with the descendants of Algol — I don’t know enough about either set of languages to comment.

I think it’s important not just that the axioms be well chosen, but that there be few of them. Mathematicians have always felt this way about axioms— the fewer, the better— and I think they’re onto something.

…

I have a hunch that the main branches of the evolutionary tree pass through the languages that have the smallest, cleanest cores. The more of a language you can write in itself, the better.

Is this why I like Smalltalk? Elsewhere (now I’m really burning time), Graham suggests that C, Lisp, and Smalltalk were, “in effect designed for good programmers, because good programmers designed them for their own use.” He also proposes, in his new dialect of Lisp, to provide programmable syntax on top of the standard Lisp s-expressions (when, oh when, will I find the time to play with Lisp?). That is, if I understand correctly, you can literally design your own language, so long as you can express it by macro expansion into s-expressions.

Languages evolve slowly because they’re not really technologies. Languages are notation. A program is a formal description of the problem you want a computer to solve for you. So the rate of evolution in programming languages is more like the rate of evolution in mathematical notation than, say, transportation or communications. Mathematical notation does evolve, but not with the giant leaps you see in technology.

This warrants some thought in the context of modelling frameworks and tools. The expression of a model is made in a language. That language shouldn’t get in the way, but should also make it as easy as possible to express any given model.

Inefficient software isn’t gross. What’s gross is a language that makes programmers do needless work. Wasting programmer time is the true inefficiency, not wasting machine time. This will become ever more clear as computers get faster.

Hear, hear. This can be taken to extremes:

There are more shocking prospects even than that. The Lisp that McCarthy described in 1960, for example, didn’t have numbers. Logically, you don’t need to have a separate notion of numbers, because you can represent them as lists: the integer n could be represented as a list of n elements. You can do math this way. It’s just unbearably inefficient.

Then some interesting thoughts on layering of software and its impact on efficiency and flexibility.

Writing software as multiple layers is a powerful technique even within applications. Bottom-up programming means writing a program as a series of layers, each of which serves as a language for the one above. This approach tends to yield smaller, more flexible programs. It’s also the best route to that holy grail, reusability. A language is by definition reusable. The more of your application you can push down into a language for writing that type of application, the more of your software will be reusable.

Somehow the idea of reusability got attached to object-oriented programming in the 1980s, and no amount of evidence to the contrary seems to be able to shake it free. But although some object-oriented software is reusable, what makes it reusable is its bottom-upness, not its object-orientedness. Consider libraries: they’re reusable because they’re language, whether they’re written in an object-oriented style or not.

Hence the success of component based programming, which throws out much of object orientation? Inheritance isn’t layering, it’s all in one place?

For example, types seem to be an inexhaustible source of research papers, despite the fact that static typing seems to preclude true macros — without which, in my opinion, no language is worth using.

That is intriguing. Graham is obviously a Lisp guy, and Lisp has macros. I’ve never used Lisp, and have no experience of using macros, and have gotten by alright. What, I wonder, have I been missing out on? Certainly this is an interesting sounding new argument in the static vs. dynamic typing debate (any efficiency issues having been dispatched by the preceding discussion).

Finally comes yet another allusion to the “tools constrain our thinking” meme, which I made an issue of in my PhD thesis.

You’d think it would be obvious to creatures as lazy as us how to express a program with the least effort. In fact, our ideas about what’s possible tend to be so limited by whatever language we think in that easier formulations of programs seem very surprising.

This ties in too with the view from semiotics that language and culture are models, and models constrain the modes of operation of our brains. I need to explore the interaction between these ideas more.

The Object Modeling System

Not to be confused with the Dutch Open Modelling System, the Object Modelling System (OMS) appears to be (yet another) modelling system/framework being developed by a collaboration of US government agencies (the Agricultural Research Service (USDA-ARS) at the Dept. of Agriculture (USDA), the Natrual Resources Conservation Service (NRCS), the Geological Survey (USGS), the (GPSR), and the Institut für Geographie at the Friedrich-Schiller-Universität, Jena (FSU)). There are at present no downloads, documents, and no anonymous CVS access, so it’s impossible to evaluate OMS. According to the web site, it’s a Java Netbeans based framework.

The Great Plains Research Unit (GPSR) obviously a) used to have some involvement with the OMS and b) used to have some association with the Colorado State University, but the GPSR history page makes no mention of CSU, and the site makes no mention of the OMS. Weird.

This sourceforge project with its homepage at Colorado State University is the same thing. The sourceforge project is empty, and the Colorado State URL times out, but the Google cache comes to the rescue.

Ideal PIM

There are dozens of personal information management applications
about. Some are more-or-less integrated sets of tools, others are
loose collections of specific applications. The more integrated
attempts such as Microsoft Outlook, Ximian Evolution,
Kroupware generally only
loosely link elements. The Open Source Applications
Foundation is working on a new PIM application called Chandler
(which I’ve mentioned
before),
which sets out to be something different. A worthy aim, but from
scanning the mailing list discussions and the architecture, it looks
worryingly as if Chandler will remain stuck in a similar model.

The basic problem to my mind is that PIM developers start out with the mindset that they want tools to do things like

• read and write email,
• manage contact details,
• maintain a calendar,
• keep track of tasks (todo list), and, if you’re lucky,
• manage documents.

Of course, all these features are indeed required, but they shouldn’t the bottom layer. Corel’s InfoCentral (which is mentioned a couple of times on the Chandler wiki), although not perfect, got this right, to my mind — although I obviously don’t know how it was implemented, it certainly appears to be the case that InfoCentral is built on a very flexible entity-and-association database.

Michael Koenecke has written a rant about why CorelCENTRAL, which claims to be a successor to InfoCentral, is nothing of the sort. It abandons precisely what made InfoCentral different, and what made those who gave InfoCentral a real chance rave about it. He provides the useful example that a telephone extension is a property of the relationship between a person and an organisation, not of the person or the organisation. Doing things this way also reduces the need for excessive complexity in the schemas for individual entities — a person can be associated with multiple companies without direct support for this in an address book component.

With this sort of approach it will be necessary to map the underlying entity-and-association model to the more mundane views of contacts, appointments, and so on to support synchronisation with handhelds or other diaries and contact databases, but to try to layer the linking on top of those views just can’t work as far as I can see.

One of the Chandler design mailing list discussions was about the need for some kind of “paperclip” feature, which allows you to tie sets of documents together. It was pointed out that a paperclip in this sense is nothing more than another name for a folder, but in any case the need for this as a specific feature just goes away in any case with the totally flexible linking model of InfoCentral. People are linked to documents are linked to emails are linked to appointments. Any of those can be used as an entry point to find the others. And if you want to create ad hoc groupings, create a new “group” entity type with no fields but a name, and then link everything to it that you want to group. In general though, those groups surely usually have more meaning than that, so could actually be instances of a more specific entity type.

Gary Yuen, who seems to hang out on the Chandler lists a lot, made a good point in the course of this discussion of paperclips:

If the question is how does data, or any object, relate to other objects, perhaps the better approach is to not examine folders, desktops, paper clips, or hyperlinks but instead to think about what was it we were trying to represent by these concepts. Instead of thinking how can we use hyperlinks, let’s think about what is the purpose of a hyperlink.

…

Was it the inventor of hypercard that said everything is related to everything else? could through the study and memory of when we create, save, or retrieve data in itself create relationship hiearchies that we currently miss through our continued use of folders or desktops?

It’s just possible that this sort of model will come out of Chandler. It is supposed to use an RDF data store, which certainly should give it the underlying flexibility. Will it be possible to create new types of entity and new types of association on the fly, though?

I still feel that what seems to me to be the higher level data model of topic maps would be a good base layer for all of this. There are issues — it must not demand that the user develops a perfect ontology of all of their stuff up front, and it must be possible to modify the schema over time, for example — but the basic structure is there. Sure it’s possible to model this in RDF as well, but going from RDF to an accessible user interface is a much bigger step than from topic maps, I would think.