On Artificial Intelligence

Some thoughts on Artificial Intelligence.

Professional and Academic Experience

I've had the surprising opportunity to intersect various topics in Artificial Intelligence at many points in my professional and academic career:

1. Past work with Microsoft Research - on Contract, they do all sorts of fun things. As mentioned before - they also host research at the intersection of Formal Reasoning, Logic, and Computer Science.
2. As a Feature/Platform/Product Software Developer/Engineer using some A.I. tools in limited ways to help automate boilerplate, address tedious and monotonous minor tasks (like patch version bumping), improve local development text search (one of the main A.I. superpowers at present IMO is the ability to perform the equivalent of a "super-GREP" command across multiple directories and code-bases while keeping a single search context - this would be equivalent to manually running tens of manual commands and then mentally combining their results).
3. Some familiarity with many basic architectures: Multi-Layer Perceptron (I wrote the original article way back when in 2017), Convulutional Neural Network, Automated Theorem Proving, empirical proof or confirmation (of hypotheses in pure maths), etc.

The Philosophical Tradition

What does the philosophical tradition have to say about certain worrying limits to Artificial Intelligence?

The philosophical tradition admonishes us that there are hard limits on justification, inquiry, proof, methodology, and technique:

1. Induction vs. Deduction vs. Abduction - these seem to be distinct forms of reasoning and are seemingly irreducible to each other.
2. Empirical (A Posteriori, Statistical, observed) vs. A Priori (Probability, reasoned) techniques/approaches.
3. Philosophy reminds us that we make use of all of these - often mixing and matching reasoning techniques to a relevant end.

Philosophy remains one of the best places to dig up future Deep Tech and interesting future industries, products, consumer goods, etc.

It's a curious and surprising state of affairs given the other historical examples that coincide with our current best consumer products/technologies:

1. Web of Belief → The Internet
2. Ready at Hand, Present at Hand → Smart Phones
3. Will to Power → Electrification (an alternative reading to be sure)
4. Eternal Recurrence → Film, Silent Pictures → Social Media
5. Language Games and Forms of Life → DNA → Bioinformatics
6. Characteristica Universalis → Programming Languages
7. The Philosopher's Stone → Nanotech
8. Runes → CPU Fabrication, laser etching
9. Golems, Greek Automata → Robotics
10. Phenomenology, Sense Data → Quantum Dot LEDS, Pixels
11. Universals → Type Theory → Object Oriented Programming, SOLID

Precursors, inspirations (or incarnations)?

Philosophy Cited in Artificial Intelligence Research

While oft-mentioned in the Press, amongst buzzy key-words, etc. few have read the actual Computer Science and/or Artificial Intelligence research papers inspiring great breakthroughs/leaps forward in the discipline.

As it turns out, Philosophy is directly cited and frequently so (in fact, during the 1980's it was one the primary disciplines that would publish nascent Artificial Intelligence research). Some influential examples (in no particular order):

1. math.stanford.edu - publications.html
2. cmu.edu - awodey.html
3. sjvickers.github.io - Ontology.pdf
4. https://csli.stanford.edu
5. www-formal.stanford.edu - ailogic.pdf
6. geeksforgeeks.org - propositional-logic-in-artificial-intelligence
7. semanticscholar.org - 33c8242e6c38623fd8896e15fe2f05a7b9f73ea2

Lesser-Known Techniques

From the above, it shouldn't come as a surprise then:

1. That many highly-pumped Deep Learning Models exhibit statistical peculariarities (low accuracy, hallucinations, etc.) - this has traditionally been held to be a hard limit on such kinds of reasoning.
2. Only the tiniest amount of research overall is driving most investment - traditional approaches are being sidelined or ignored.

Here are two that I think should be looked more closely at.

Set Theoretic Predicates (Again)

Patrick Suppes' Set Theoretic Predicate technique described in Representation and Invariance of Scientific Structures.

This technique basically involves transforming and parsing lengthy social science theories (academic papers) into succinct algebraic structures - a 15 Page paper can be succinctly expressed in 4 or 5 sentences of Predicate Calculus (equipped with various operators, Sets, etc. as need be). A mathematician or computer scientist immediately grasps that one can then use existing formal reasoning techniques to determine consistency, inaccuracies, compare with other theories, and generally reason (and more importantly arrive at novel conclusions that aren't explicitly stated in the paper).

For example: a survey data table is trivially parseable into formal statements that can then be used in basic logical reasoning (about that specific empirical domain). Basic validation checking can be used to verify the accuracy of these statements/theories.

(I successfully proposed this to the Military a long time ago to help address the Replication Crisis and the technique has advanced in use through more mature academic institutions. I think the technique can be used more broadly - especially in tandem with the somewhat cumbersome document understanding systems that presently abound that merely summarize the contents of a file using Natural Language.)

Markov Chains and Logic Learning Networks

At one point Markov Chain-based Artificial Intelligence was all the rage (like the early 2000s - this brings back memories of AOL and dial-up modems).

There are two key ideas here:

1. More recent architectures combine the outputs of other techniques (the success of GAN is partly based on such an approach too albeit with Generators, Discriminators). The combination of Artificial Intelligence techniques may allow for the weaknesses of one approach to be mitigated or strengthened by another (obviously, we use both Inductive and Deductive reasoning - and that fact may very well be a key to understanding Abduction which use perhaps most prominently). This approach is common with Markov Logic Learning Network-based approaches and is something I've mentioned before.
2. Markov Chain-style semantical systems can trivially combine or permutate variations of key concepts to generate novel ideas (like a Madlib-style generator with keyword constraints). In fact, one or two early ideas of mine were inspired by this very approach (the idea for Non-Eternalism about Logic and Semantic Immuration were inspired by a simple combinatorial tool I built back in 2016 that generated what are effectively "prompts" for greater human input and development - my own)!
3. Vectorization approaches convass the sum totality of a vast data set for appropriate symbol/token combinations often generating elaborate but occasionally inaccurate summaries.
4. Combinatorial approaches may allow for "greater creativity" and as yet remain somewhat unexplored (particularly w.r.t. their interactions with GAN, CNN, and other Deep Learning techniques). By "greater creativity", I mean the generation of novel scientific, engineering, mathematical, and philosophical theories/ideas (exceeding the mostly "artistic creativity" and "textbook recitation" extant systems have thus far demonstrated).
5. Again, the two might be more fruitfully joined together to help associate all kinds of syntactic and semantic patterns, constraints, and relationships between various markers. Prior Markov-style systems required all of the relevant associations to be present prior to generating an (accurate, useable) solution (defined probability space often with weights) - using Vectorization summary techniques we might be able to optimize this for incomplete or partial data sets.

I briefly skimmed the internet to see if that was a completely bonkers idea and found some recent researchers looking into this space:

1. lamarr-institute.org - incomplete-data-markov-random-fields
2. mdpi.com - 2542