One of my users wrote to me a couple of days ago, pointing out that MQL statements like the following:
SELECT ALL OBJECTS WHERE [sentence NOTEXIST [Phrase] [Clause FOCUS] ] GO
would not return any FOCUS'ed clauses.
I looked into it, of course, and found some deeper issues, which I'd like to explain.
1) First, what does the query mean?
Well, let's start by looking at what its counterpart without the non-existence would mean:
SELECT ALL OBJECTS WHERE [sentence [Phrase] [Clause FOCUS] ] GO
It would mean something like the following (in First-order Predicate Logic):
exists S in Sentence restricted to Su: exists P in Phrase restricted to S: exists C in Clause restricted to S: P.last + 1 = C.first
What I mean by "Sentence restricted to Su" and "Phrase restricted to S" is that we are after all those S and P which belong to the given object type, but which are selected from the subset that is wholly c0ntained in the set of monads being mentioned (Su and S).
This is the standard MQL interpretation, laid down by Doedens in his 1994 PhD thesis.
Now, let's see what this means when you add the NOTEXIST. We start by adding a little, innocent "not" in front of the "exists P" clause:
exists S in Sentence restricted to Su: not exists P in Phrase restricted to S: exists C in Clause restricted to S: P.last + 1 = C.first
Standard mathematics operations are then applied:
exists S in Sentence restricted to Su: forall P in Phrase restricted to S: not exists C in Clause restricted to S: P.last + 1 = C.first
and again;
exists S in Sentence restricted to Su: forall P in Phrase restricted to S: forall C in Clause restricted to S: not (P.last + 1 = C.first)
As you can see, what it really means is that there exists some S, inside of which all phrases and all clauses stand in a relationship where no phrase P ends at the monad before the first monad of any clause C.
(!)
That would be a pretty darn interesting analysis of a sentence. Overlapping boundaries would abound.
2) Second, what did I do about it.
Well, I decided to dictate that NOTEXIST could only be applied to one object block (that's as it is now anyway), and that that object block must be the only block present in its blocks.
Not content with simply fixing that, however, I decided to be adventurous and redesign the implementation of the language from scratch.
That is now completed, and the end result is quite pleasing:
- NOTEXIST works correctly now, as per the above dictum.
- You can now have the Kleene Star on the first object block in a blocks, AND also on the first one after a power block "..". This allows you to make something optional at the beginning of a blocks, which was what NOTEXIST was being used as in the above example anyway.
- You can now have a gap_block at the beginning of a blocks.
- It is only about 4% slower than the previous implementation, and I think I can tweak that...
- Not only do all regression tests pass beautifully, but some correctness issues that had been present for a long time are now resolved. In particular, NORETRIEVE now works as designed, and doesn't cause incorrect results to be retrieved.
- The code is MUCH simpler, MUCH more maintainable, and much more beautiful than the old code, which had grown somewhat crufty.
This will make its appearance in the next public release after 1.2.0.pre160.