# Our Inner Screen Vector Space

I’d planned a different first post for May Mind Month, but a recent online conversation with JamesOfSeattle gave me two reasons to jump the gun a bit.

Firstly, my reply was getting long (what a surprise), and I thought a post would give me more elbow room (raising, obviously, the possibility of dualing posts). Secondly, I found the topic unusual enough to deserve its own thread.

Be advised this jumps into the middle of a conversation that may only be of interest to James and I. (But feel free to join in; the water’s fine.)

@James: “So one configuration of the 500 neurons can represent ‘green’, and a different configuration can represent ‘ball’, and a different configuration can represent ‘base’ (as in baseball base), and a different configuration can represent ‘green+(base+ball)’, or ‘green baseball’.”

Let me see if I’m following…

In Neuron Space 500 each axis (neuron) has five values indicating status. The idea is that there is a symbol, {GREEN}, with a specific firing pattern and thus a vector in the space. There is also a {BALL} symbol with a different pattern. And so on, such that any mental symbol has some vector in the space.

Then, by adding the {GREEN} and {BALL} vectors, the resulting {GREEN+BALL} vector is meaningful.

Correct assessment?

The vector for {GREEN+BALL} is just a vector in the space, right? Are you saying there is vector math that adds {GREEN} and {BALL} to get that {GREEN+BALL} vector?

Or just that it, too, is vector?

I’m not at all sure the combination of {GREEN} firings with {BALL} firings is the same as {GREEN+BALL} firings.

In fact, I’d be inclined to doubt it. Why would there be any connection?

It seems like there’s a mapping problem between neurons firing and concept symbols.

§

Here’s an example that might explain what I mean:

My SS#, name, full address, home and cell numbers makes about 130 bytes. Given people with longer names or addresses, UTF-8 encoding for international, 150 bytes seems reasonable for a basic person record.

That is, there is enough “elbow room” that we can easily encode meaningful (human-readable) patterns in 150 bytes that link everyone on Earth with some basic personal details.

Each address is a vector, of course, in our imagined 150-byte Memory Space.

Every combination of byte values in those 150 bytes is a vector in Memory Space. Most most of them are gibberish. Even the small set that use legal characters in the right places (e.g. numbers for phone) will have more invalid records than valid ones. Even ones that look okay might not exist.

So we can start off with 256150 vectors but the useful ones are only a tiny subset. (And I do mean tiny. One-trillion records is only 2565 — five bytes.)

More to the point consider that we cannot create meaningful new records by combining the vectors for existing records.

The vector for the house on the left and the vector for the house on the right can’t be combined to create the vector for the middle house — you’d end up with a gibberish vector.

That’s because the vectors of this space depend on the bits of RAM — similar to how vectors in Neuron Space depend on neurons firing. As such, the vectors have no semantic connection to the symbols they represent.

§

Another issue for the idea of vector combination is that if Neuron Space 500 contains all mental concepts, what determines which concepts are “atomic” and which are “combinations?”

Why would {GREEN+BALL} be a combination of {GREEN} and {BALL} symbols rather than an atomic symbol referring to a specific green ball I have in mind?

What determines the “basis” vectors (the vectors that are “pure” and not combinations of any other vector) for the space?

§

The idea of vector combination aside, there is still the problem that 500 neurons with five degrees of freedom results in a Neuron Space where each vector can be expressed in 150-bytes.

Because:

5500 = 3.055 × 10349
21200 = 1.7218 × 10361 (bigger!)
1200 bits ÷ 8 bits/byte = 150 bytes

And, as I’ve said, I think individual mental symbols are far richer than five neurons can encapsulate. Concepts of reality — even momentary ones — seem to me vastly richer than 150 bytes can manage.

To be clear, here I’m not saying the vector space isn’t big enough for all the concepts (although I think that, too). I’m saying individual symbols are too big for the vector space.

I don’t think 150 bytes is enough even for song or image fragments.

Not even as symbols. The problem there is the symbols are low-level. The moments of a song, the small circle of visual attention, these aren’t symbols in themselves — they’re comprised of symbols, so any real mental concept is a package of symbols.

§

I also see an issue of richness.

My mental concept of {GREEN} isn’t a single vector, but a collection of many to accommodate all my many senses of {GREEN}. There are, likewise, many vectors for {BALL}.

I don’t think it’s realistic to think of {GREEN} as a vector at all, but as a region in vector space comprised of all the vectors that have {GREEN-ness} as a property.

As I suggested up top, I think there’s a mapping problem. In a space built on neurons firing, the basis vectors aren’t based on mental concepts — they’re based on which neurons happen to be firing.

In order for a {GREEN} vector to make sense relative to a {BALL} vector, the basis vectors — the atomic axes of the space — have to be mental concepts, not just whichever bits happen to be active.

That’s the only way the space is conceptually unified. Otherwise it’s just random vectors with no connection to each other.

§

Finally, I might be wrong, but I get the sense Neuron Space is akin to RAM such that a given vector can be seen as different symbols in different contexts?

If so, I’m not sure that matches my (admittedly scant) understanding of brain physiology.

At least, I’ve never heard of anything like link swapping. I’d have to see a neurophysiology reference providing a basis for the idea for me to say more.

(And maybe I got that wrong, and it’s not part of the idea.)

§

My bottom line:

1. I don’t see how vectors are useful, especially when based on neurons firing.
2. I don’t think 500 neurons (150 bytes) is sufficient for symbol representation.
3. I don’t think 500 neurons is a big enough vector space for all the symbols it has to map in a given context.
4. I don’t think attention is anywhere near as simple as a single vector swinging around in some symbol space.
5. Conscious attention seems more parallel than this allows.
6. Doesn’t account for glial cells and other potentially contributing systems in the brain.

What I can imagine is what I thought this was headed for originally: more a whole brain thing.

I can imagine a set of parallel vector spaces, each a conceptual space with its own concept vector, and consciousness visualized as the operation of all active vectors.

But it’s still just a metaphor. I’ve never understood what the point of the vectors was other than as a metaphor or description.

The canonical fool on the hill watching the sunset and the rotation of the planet and thinking what he imagines are large thoughts. View all posts by Wyrd Smythe

#### 55 responses to “Our Inner Screen Vector Space”

• SelfAwarePatterns

I know there’s been a lot of prior discussion on this (which I haven’t followed closely). So apologies if this ends up being utterly irrelevant.

Antonio Damasio has a concept he calls covergence-divergence zones (CDZs). The idea is that mental concepts are registered in the brain when neural firing patterns converge into a particular region. Although it’s crucial to understand that the mental concept is not fully contained within the CDZ, but within the vast hierarchy of neural firings that converge on it.

So we might have early visual neurons reacting to certain shapes, contours, colors, textures, etc. As the signals propagate up, the neurons become progressively more selective in what will excite them. Maybe there are neurons that only fire for, say, a hairy quadruped. Successive neurons might only fire it’s specifically a bear, or a dog, or a wolf.

Each of these later points could be considered a CDZ, which feed other CDZs further up from the early sensory regions. Eventually you might get to a CDZ that only fires for a particular dog. One one that only fires for a particular human (ex- Jennifer Aniston).

But each of the inputs going into a CDZ are effectively loaded with meaning. They symbolize something, a meaning built out of the entire hierarchy of firings.

Everytime James talks about symbolic inputs, I think of Damasio’s CDZs. It seems like there may be some consilience between these views, although Damasio’s is much more specifically embedded in the actual biology, and James’ seems more like a computer science interpretation of what is happening.

• Wyrd Smythe

“Eventually you might get to a CDZ that only fires for a particular dog. One one that only fires for a particular human (ex- Jennifer Aniston).”

That does seem to reflect how our biological network operates. Does Dammasio say how many neurons are involved in CDZs? Are they’re located in different areas rather than some single inner “staging area”?

It sounds like they represent the regions of neural net phase space that contain specific learned concepts (my dog, Ms Aniston, etc).

I’ve been thinking a little about grid cells lately, how they represent a physical 3D space as a literal physical map. Again, the brain seems very representational.

Real world spaces literally mapped to brain spaces. Makes sense real world concepts would be physically mapped as well. That’s always been my understanding.

“But each of the inputs going into a CDZ are effectively loaded with meaning.”

I would think so. As with a neural net, those “atomic concept” areas would lie at the (virtual, effective) “top” of a hierarchy of levels of CDZs with the lowest levels processing raw sensory inputs.

There is, perhaps (or is there), a separate question about how those areas are accessed if I remember my dog or Ms Aniston rather than see them. Where do the inputs come from in the second case?

Memory, obviously, but you see the possible problem. Where and how do sensory inputs converge with memory inputs if both end up stimulating the high-order CDZ?

“Everytime James talks about symbolic inputs, I think of Damasio’s CDZs.”

I’m afraid I don’t see the connection you’re making? Is it about the symbols?

“James’ seems more like a computer science interpretation of what is happening.”

I don’t see much CS to it (other than using symbols). CS is about computation, and the vector space idea is more mathematical.

To me it seems to be about the ability to combine vectors into meaningful new combined vectors, but, as I showed, they don’t work that way. That only works in terms of meaningful basis vectors.

• SelfAwarePatterns

The CDZs are very much seen as a physical location. They are located in CDRs (CD regions). Damasio said that the number of CDZs would be in “the many thousands” but there only being a few CDRs, with multi-modal concentrations in the superior parietal lobe.

I don’t recall him ever explicitly discussing how many neurons might be involved. The Jennifer Aniston neuron seems to imply it might come down to one in some cases, but I would think most might be a cluster of them.

A key feature of this idea is that the neural hierarchies that makes up a concept can be activated in reverse by stimulating the CDZ. So when we remember something, we actually activate the same neural hierarchies that initially perceived it, albeit with less vividness since there isn’t sensory information coming, so the very lowest layers probably don’t get retro-activated.

What might stimulate a CDZ? Another CDZ with connections. Or signals from outside of the current CDR. Imagination may be CDRs in the frontal lobes signalling CDRs in the sensory cortices to reverse activate mental images in particular sequences.

“I’m afraid I don’t see the connection you’re making? Is it about the symbols?”

Each input to a CDZ could be interpreted as a symbol. So there might be a synapse that indicates a quadruped, another that indicates a certain size, yet another that indicates a snout shaped face, and other attributes that might cause a dog neuron to fire, although slightly different weights may cause a related wolf one to fire instead. Or maybe they both fire and we spend a minute trying to discriminate.

That’s my interpretation anyway. It may not entire match James’. And I’ll leave him to defend his vector ideas.

• Wyrd Smythe

“The Jennifer Aniston neuron seems to imply it might come down to one in some cases”

I’ve always wondered about that idea. So many other neurons are firing to converge on that one putative neuron… how can it be picked out as The One?

I’m far more inclined to believe all specific memories are distributed. There is no The One neuron.

I base this on how these kinds of networks work in general. Nodes always affect adjacent nodes. Having a “Jennifer Aniston” neuron implies a strong isolation from all adjacent neurons (or even distant neurons being fed similar data).

There’s also a numbers thing. Given the billions of neurons, just one is for something as complex as Jennifer Aniston? Surely not.

You know how holographs work, right? My understanding is that the brain is holographic — information is distributed over the network. (As with DL NNs.)

“A key feature of this idea is that the neural hierarchies that makes up a concept can be activated in reverse by stimulating the CDZ.”

Our neural nets are strictly unidirectional, so how does that work?

It seems that somehow memory has to feed into the same higher network that sensory data does.

“Each input to a CDZ could be interpreted as a symbol.”

Sure. (Okay, so it was the symbols. 🙂 )

• SelfAwarePatterns

I think in the case of the Jennifer Aniston neuron, it wasn’t just one neuron firing, it was that only Jennifer Aniston excited that particular neuron, and pictures of her from various angles and her name all worked, indicating it was multimodal. But absolutely Aniston isn’t represented in only that neuron, but in the hierarchies of neural activation that converge on that one neuron. There’s nothing magical about that neuron. If it was removed, another might take up its role, although possibly after some adjustments from the overall network.

On CDZs and how retroactivation works, I’ve wondered about that myself. If you understand synapses, you know they’re not bidirectional, so there have to be feedforward and feedback connections. The feedback connections, even when we’re not explicitly remembering something, probably help to solidify a particular perception (or misperception as the case might be).

“It seems that somehow memory has to feed into the same higher network that sensory data does.”

The thing to understand is that when you remember something, you reuse the same sensory structures that did the initial perceiving. You do it with retroactivation, coming from the CDZs down to earlier sensory regions, as opposed to from the sensory regions to the CDZ.

(In truth, even perception involves a lot of retroactivation feedback. We perceive what we expect to perceive, with sensory data only fine tuning rather than totally defining our perception.)

For some diagrams on CDZ activations: http://willcov.com/bio-consciousness/diagrams/Damasio%20CDR%20diagram.htm

• Wyrd Smythe

“But absolutely Aniston isn’t represented in only that neuron, but in the hierarchies of neural activation that converge on that one neuron.”

Ah, yes, this makes much more sense. My next question is whether that was the only such singularly activated neuron, or if there were others that only lit up for Ms Aniston. From what you said, it does sound like the neuron was only activate for JA inputs, but not for any others.

For the rest, now that we’re talking about it, yeah, of course there has to be wiring in both directions. One of those, ah, yes, pretty obvious now that you mention it. 🙂

• SelfAwarePatterns

Them using the singular “neuron” probably indicates it was the only one they saw, but I’m sure they can’t say that categorically, or even that the particular neuron is never ever excited by something they didn’t put in front of the patients. It sounds like the technique to see this involves electrodes, which usually only happens in humans as part of some medical procedure that the researchers then piggyback on. So the section of brain they were watching must have been very narrow.

• JamesOfSeattle

Regarding the Jennifer Aniston neuron, my working hypothesis is that the unit of cortical memory is the cortical column. This from Wikipedia:

Columnar functional organization

The columnar functional organization, as originally framed by Vernon Mountcastle,[9] suggests that neurons that are horizontally more than 0.5 mm (500 µm) from each other do not have overlapping sensory receptive fields, and other experiments give similar results: 200–800 µm.[1][10][11] Various estimates suggest there are 50 to 100 cortical minicolumns in a hypercolumn, each comprising around 80 neurons. Their role is best understood as ‘functional units of information processing.’

An important distinction is that the columnar organization is functional by definition, and reflects the local connectivity of the cerebral cortex. Connections “up” and “down” within the thickness of the cortex are much denser than connections that spread from side to side.

So based on the above, I think I’m talking minicolumn. My current ill-informed understanding is that each column sends one axon to the thalamus. For the Jennifer Aniston column, that one could be the “Jennifer Aniston neuron”. But I could also see a Jennifer Aniston hypercolumn, in which case the multiple axons to the thalamus together may facilitate generating the “Jennifer Aniston vector” there.

As for what might stimulate a CDZ? Maybe a reciprocal connection from the thalamus, possible routed through hippocampus?

Question for Wyrd: If the concept of “Jennifer Aniston” is distributed holographically, why would poking one spot invoke a specific concept and not a multitude?

• Wyrd Smythe

“If the concept of “Jennifer Aniston” is distributed holographically, why would poking one spot invoke a specific concept and not a multitude?”

Oh, there’s definitely localization, we know that. But I don’t believe in single neurons.

This Jennifer Aniston Neuron (let’s call her JAN), it fires or doesn’t fire, the firing rate changes indicating a magnitude, and some think the shape of the pulse might matter, but it’s still a limited amount of information.

So,… the JAN reacts every time I think of Ms Aniston, or just when I think about her in certain contexts? Does it react if I dream of Ms Aniston? If it reacts in every possible Aniston context, does it never react if she’s not involved?

If I watch a movie she’s in, but isn’t appearing on screen at the moment (but might), does the JAN react?

I just don’t think the human brain is that precise. It’s a soup (an ocean, more like) of mental concepts and fragments.

• JamesOfSeattle

Mike, I absolutely agree about the CDZ’s. And by my definition they describe smallish conscious agents within the cortex. But (I hypothesize), relative to the grand conscious agent (Damasio’s autobiographical self), these CDZ’s are pre-conscious. They build up concepts and then (if selected by the attention mechanism) inject them into the autobiographical consciousness by generating a symbol (hashkey) in the (appropriate thalamic) “500 neuron space” we’ve been talking about. That hashkey, I now for the first time propose, points back to the CDZ in the cortex, i.e., can re-activate that CDZ under the right circumstances.

*
[maybe getting too comfortable with my wild speculation hat]

• SelfAwarePatterns

James,
What specifically do you mean by “hashkey”. Are we talking a string of values like in tech computer? Or a pattern of neural traffic? (Or a pattern of neural traffic that can be interpreted as a sequence of values?)

• Wyrd Smythe

Given the parallelism of the brain, such a key can be seen as the parallel firing of all neurons involved. That key is the vector James speaks of — the moment-to-moment content of the “inner screen.”

As I understand it… 😀

But, as a general thing, complex keys can be processed in parallel on systems with a very large “data buss.” Just think of it as a huge digital word. (In fact, the 150-byte number I’ve been mentioning.)

• JamesOfSeattle

Mike, I was using the term Wyrd used in a different reply. The more apt idea is a memory pointer (which is where Semantic Pointer comes from), but that’s pretty much what the hashkey is, right? But in this case, the pointer is instantiated by a pattern of neural firing in a particular set of neurons. Don’t know if I would call the firing pattern “traffic”. In the model, the pattern is thought of as representing a vector. So any given “vector” can be a pointer to a place in the cortex (I think, maybe), and the vector counts as a symbol which means the “concept” associated with the appropriate cortical structure that it points to.

Clear?

*

• Wyrd Smythe

“…that’s pretty much what the hashkey is, right?”

A pointer is an address in memory; it refers to a specific location and has no necessary connection with the contents of that location. (That is, the contents of the location could change and the pointer would still point to that location.)

A hashkey is mathematical operation over a “file” (collection of bytes). It has no connection with the file’s location, but is a “unique fingerprint” of the contents. In this case, if the data changes, the hashkey is no longer valid.)

(It looks like I do understand what’s said to be going on.)

• SelfAwarePatterns

Thanks James. It’s still not clear for me. I understand the concept of a pointer, but in the context of a memory address to a particular location, a paradigm that the brain shows no sign of having. The prefix “semantic” might mean this type of pointer works on different principles?

I guess I’m not clear how this translates into neural firing where accessing a particular location means exciting the right pathways. I’m a little worried that “pattern of neural firing” is making a withdrawal from a fund that requires more explanatory deposits. (Or I’m just confused.)

• Wyrd Smythe

“I understand the concept of a pointer, but in the context of a memory address to a particular location, a paradigm that the brain shows no sign of having.”

The link James pointed me to says (bolding mine):

The term ‘Semantic Pointer’ was chosen because the representations in the architecture are like ‘pointers’ in computer science (insofar as they can be ‘dereferenced’ to access large amounts of information which they do not directly carry). However, they are ‘semantic’ […] because these representations capture relations in a semantic vector space in virtue of their distances to one another, as typically envisaged by connectionists.

So they mean programmer’s pointer in the sense of pointing to a larger data chunk, and they also mean “pointer” as in “arrow” as in vector. The “semantic” part comes from having an abstractly defined semantic space (space of concepts) rather than a space of numbers.

The page mentions “…Semantic Pointer Architecture (SPA) and its use in cognitive modeling.” Which suggests this is a descriptive technique — a way of talking about semantics.

It goes on to list three operations that can be performed on these vectors: superposition, binding, and unbinding. The first seems to be some form of vector addition (in one place it’s suggested it’s just member-wise addition). I’ve never heard of the latter two operations in connection with vectors.

The page itself appears to be the support page for a Python package that implements these “semantic vectors.” From what I can tell it allows you to play around with simple models, although I’m not entire sure what the models are supposed to be.

“I guess I’m not clear how this translates into neural firing…”

Not clear here, either. I can say James and I have been discussing three distinct topics.

One is the semantic pointers vector space thing, which I see so far as fine but purely descriptive, a way to talk about (or crudely model) our semantic phenomenal content. I don’t see any direct application to our brains or minds.

Two is the idea of an “inner screen” — the 500-neuron package that [displays? controls?] the spotlight of our consciousness. I have a number of objections here, many related to the 500 neurons — a number I find vastly too small. And I’m not sure I buy into the whole thesis.

Three (not touched on here) is the idea that an Input-Process-Output module (such as a thermostat or algorithm) is (A) actually honest-to-gosh conscious, (B) a unit of consciousness (whatever that means), or (C) not conscious at all. You know me well enough to know where I fall. 🙂

I’ve asked James to table that last topic for a separate thread. That one is a War of Axioms, but I’m willing to entertain arguments.

• SelfAwarePatterns

Thanks for the explanation. If the semantic pointers do in fact map to connectionist understandings, I could probably work with that.

On the “inner screen” idea, the intuition that consciousness lives in a particular spot in the brain seems to be a very powerful one. A lot of people have it in their theory of consciousness. Some see it in the midbrain region, others in the thalamus, some in the striatum or basal ganglia in general. I read one theory a while back that it lives in the hippocampus.

The problem is that the evidence, as I understand it, doesn’t line up with any of these conceptions. Out intuition of a conscious system seems to come from observing a number of capabilities interacting with each other. No one capability by itself appears to be the one and only one that provides subjective experience, nor does the absence of any one capability knock it out. And it’s only the capabilities that can be associated with networks of specific physical locations.

• Wyrd Smythe

“Thanks for the explanation.”

NP. That there’s a Python implementation intrigues me a bit, so when I get a chance I may go back and look at the API and docs. I’m little curious about how the pointer operations they listed are actually implemented.

I agree there’s nothing wrong with the basic idea of semantic vectors. Heck, I just wrote five posts in April involving the vector spaces idea (Neapolitan, Baskin-Robbins, SF-F, 2D in general, and 3D cubes) not to mention it was the topic of one of my first posts.

So, yeah, I love the basic idea (have used it for many decades). But I don’t think the neuron firing space is probably decoupled from the semantic space. I don’t think NS-500 works the way James thinks it does.

“I read one theory a while back that it lives in the hippocampus.”

Three doors down from there. In the blue house. You’ve probably driven by it many times. 😉

“No one capability by itself appears to be the one and only one that provides subjective experience, nor does the absence of any one capability knock it out.”

Exactly. And hence my intuition that consciousness is holistic and irreducible. Whatever it is, it appears to be a property of the entire system.

As you say, it can be corrupted and damaged in all sorts of ways (I’ve always enjoyed the mild corruption of the ethanol molecule, for instance), but there’s no The One.

I’ve related the story about losing Cameron Diaz in my brain for about a year or so. Just could not remember her name. Finally retrained another part to do the job and the problem vanished. I’m thinking a tiny stroke killed a collection of neurons — the top-level CDZ for Cameron Diaz, so to speak.

(My mom had several strokes in her old age, and it was fascinating (and heartbreaking) to see what speech and motion capabilities came back and which didn’t. My dad died of Alzheimer’s so I watched that consciousness alter and vanish over time. Even more heartbreaking.)

So if this 500-neuron area was as important as “inner screen” suggests, surely we would have observed catastrophic results from minor stroke victims, accident victims, possibly even drug effects. So I have to believe anything that localized and that specialized would have been identified long ago.

• SelfAwarePatterns

“And hence my intuition that consciousness is holistic and irreducible.”

Or our intuitions about it are holistic and irreducible. But I actually don’t care for the “holistic and irreducible” language. To me, it implies strong emergence, which I don’t buy. I do buy it in a weaker epistemic sense, but in that sense the phenomena can always be reduced.

I actually think silent minor strokes cause a lot of cognitive damage that we don’t realize. This kind of damage is likely increased by alcohol consumption and other types of drug use, as well as head contact sports. I suspect most humans have micro-lesions, but as long as they’re not pervasive, our brain works around them.

“surely we would have observed catastrophic results from minor stroke victims, accident victims, possibly even drug effects.”

This is actually why so many people focus in on the midbrain / upper brainstem region, such as the colliculi, reticular formation, etc. Even minor damage there tends to be devastating. The problem is all of the capabilities that can be isolated to that region appear to be non-conscious or pre-conscious, such as eye saccades and other reflexive reactions. It appears to have a vital role in arousal, which is one of the things people mean by “conscious”, but not in awareness that can be reached through introspection.

The thalamus is a more plausible candidate since too much damage to it can snuff out the awareness form of consciousness. But again, only going through the thalamus doesn’t in and of itself seem to contribute directly to awareness. But it also has vital supporting roles as a communication hub and nexus of attention processing, among other things.

• Wyrd Smythe

“Or our intuitions about it are holistic and irreducible.”

Ha, yeah, sure. 🙂

Clarifying: Do you see strong emergent properties as beyond physics and weak ones as not? Is that the main distinction?

How about physics explaining how it emerges, but seeing it as too complex to ever fully understand? (Perhaps even in a Gödel incompleteness sense or a Heisenberg sense.)

Weather, for instance, isn’t mysterious in the physics sense (there are still some things about lightning and hurricanes and tornadoes and etc we don’t know, but we’ve got a pretty good handle on most of it in terms of physics).

But weather is, perhaps, too complex to ever be fully understood. We might never be able to build our own weather (SF aside). With weather it seems obvious you need a vast physical system. Maybe our evolutionary history and brain complexity is that system.

“This kind of damage is likely increased by alcohol consumption and other types of drug use, as well as head contact sports.”

And age! Certainly I’ve explored the space of “things that are hard on my brain” pretty well over the years. I don’t begrudge it losing an actress or two.

(I’ve always agreed with the sentiment that one should slide into the grave all but used up and exhausted, drink in one hand, chocolate or cigar in the other, shouting, “Man! What a ride!!” 😀 )

“The thalamus is a more plausible candidate since too much damage to it can snuff out the awareness form of consciousness.”

And the thalamus is where James places his inner screen.

• SelfAwarePatterns

I see strong emergence as an ontological assertion and weak emergence as an epistemic one. So, to me, weather is weakly emergent. In principle, the activity of the weather system reduces to its components, although due to chaos dynamics we will never be able to perfectly predict a short term outcome, or a long term outcome in any practical manner.

But strong emergence is the assertion that something new comes into being. It’s not necessarily supernatural, but it’s something that can’t be reduced to its components, even in principle. People claim to see examples of strong emergence all the time, but in the cases I’ve investigated, it’s more weak emergence.

I’m personally not a big fan of reaching for emergence when trying to explain something. If we’re relying on it, and only it, then I don’t think we understand the phenomenon we’re talking about yet.

• Wyrd Smythe

Got it. Makes sense.

How do you see laser light emerging from physical systems whose components show no trace of coherent photons? Do the physical laws that give rise to it make it weak even though one can argue “new” coherent photons arise? Or that the coherency itself is a new emergent phenomenon?

(There’s a reason I think it’s a really good analogy!)

• SelfAwarePatterns

For lasers, maybe I need to refresh my understanding of how they work, but I’m not sure where the strong emergence might be. My understanding is that the coherence is driven by how the lase medium is excited. It’s all about causing electrons in a particular chemical element to emit photons and using mirrors to shape the beam. No?

• Wyrd Smythe

You’ve got it right about lasers. I’d only add that the “stimulated emission” part of how the material behaves is crucial to getting it to lase.

I was just probing what you meant by “strong emergence is the assertion that something new comes into being.” The coherent light is, in some sense, new to the lasing material.

So, for that matter, are the photons.

• SelfAwarePatterns

My language was incomplete. What I meant was that strong emergence seems to imply that something new comes into being which can’t be accounted for in terms of its constituents and interactions, not even in principle.

I’ll be the first to admit that I don’t know how photons emerge from an electron falling to a lower energy orbital, but my understanding is that quantum physics can account for it.

• Wyrd Smythe

Got it. Okay.

As a data point, when I debate this stuff, unless otherwise clearly stated, I’m arguing in a context of physicalism and weak emergence (per your definition, which I agree would include lasing and, for me, consciousness).

I don’t know how the photons arise, either, other than “that’s just how nature works” and I’m likewise ignorant about how consciousness arises.

• JamesOfSeattle

Wyrd gave an excellent summary of the three topics under discussion, and I thought I would take this opportunity to explain how they are related. But I need to go in reverse order, so, bottom to top, because the top requires the bottom.

3. (Using Wyrd’s terminology) Input-Process-Output model. For philosophical reasons, everything that happens can be described with an IPO model. Consciousness is about a certain subset of those things which happen. The description of that subset constrains all three parts of the model individually. (Let me know if I need to go thru those constraints again.) A process which meets the minimal constraints can be called a psychule. A psychule is a unit of consciousness. The “consciousness” of a system is the set of psychules of which that system is capable.

2. To be useful, we should be able to apply the IPO model to human consciousness. When discussing consciousness, most people are referring to a particular set of activities referred to by Damasio as the autobiographical self. As you both have noted, many philosophers have considered an inner screen to be part of this system. What would be the role of an inner screen in the IPO model? It would be Input. If that’s all there was to it (input, but no output), there would be no conscious process, so no consciousness. But what if there was a general purpose input? Like a screen, or a blackboard, or workspace, that could be accessed by multiple processes? So several IPO processes, with the Process (or what I call mechanism) located in various other places, but all having access to the same input. The Semantic Pointer Architecture explains how that screen can be implemented using a smallish set of neurons.

1. Conceptually, that screen can be thought to represent a multidimensional (indeterminate, but we’re using 500 for brevity) vector. Various vector-type operations can be performed on that vector, like binding and unbinding, producing new vectors. These operations can be performed using bio-plausible neurons as input. These vectors act as pointers because they can be used to access the underlying structures. I personally think the pointer concept is apt because I conjecture the vector/pointer can be used to reactivate specific locations (cortical columns) in memory, but I don’t think Eliasmith has said that. But what Eliasmith has done is built a neural-network model called SPAUN, implementing this functionality. SPAUN is a single network (but with sub-network organization), so, just simulated, biologically plausible neurons that can perform multiple kinds of tasks involving various kinds of pattern recognition and output via a (simulated) mechanical arm.

*
[and again, I’m saying multiple screens, in the thalamus, accessible by the PFC, and the hippocampus, and the amygdala, etc.]

• JamesOfSeattle

Additional note. The catastrophic effects of damage to the thalamus are well documented. But that gives me a thought. If my multi-screen conjecture is right, a very localized lesion in the thalamus might remove just a single category of consciousness. Where are lesions that cause blindsight? Hemi-neglect?

*

• Wyrd Smythe

The thalamus is, as Mike said, “vital supporting roles as a communication hub and nexus of attention processing, among other things.” There’s no question it’s important. I just question the “inner screen” thing, and I really question the 500-neuron thing.

• SelfAwarePatterns

Generally blindsight is associated with damage to the early parts (V1) of the visual cortex in the occipital lobe. (Hence the term “cortical blindness”.)

One of the old theories for blindsight was that the signal was bleeding over in the thalamus while being relayed through the lateral geniculate nucleus. But now we have strong indications that it happens due to processing in the superior colliculus in the midbrain, which sends a signal through the thalamus to the amgydala.

Although I suppose if the optic nerve pathway was damaged anywhere after the branch off to the superior colliculus, it could lead to the same effect since transmission to the visual cortex would be prevented. So if the lateral geniculate nucleus in the thalamus was lesioned, it might lead to the same effect. Maybe. A functional but signal deprived V1 might change the dynamics.

• Wyrd Smythe

It occurs to me that the idea of an “inner screen” with high-level symbols ({GREEN} or {BALL}) that stand for complex meanings elsewhere in the brain has some parallels with the idea of hash keys that stand for much larger files.

So when talking about what symbols (aka hash keys) can be represented in the inner screen vector space, it might be useful to think in terms of how hash key space works. (Which would be a very CS thing to do.)

The interesting points would be key collision, how keys are created, and the sparseness or richness of key space.

• JamesOfSeattle

This is on the right track. See the link I (tried to) post.

• Wyrd Smythe

I fixed the link and took a quick peek. It’s pretty much what I thought, but I’ll have to go back (when I get a chance) and look more at that binding and unbinding.

At this point, FWIW, I can’t say anything has changed my analysis. The vectors seem more like a descriptive device and I still think 500 neurons isn’t anywhere near enough (by many orders of magnitude … I’d be talking millions).

• JamesOfSeattle

OK Wyrd, let’s work through your list.

1. I don’t see how vectors are useful, especially when based on neurons firing.

You really (Really, REALLY) need to read this Introduction to the Semantic Pointer Architecture. Then get back to me.

4. I don’t think attention is anywhere near as simple as a single vector swinging around in some symbol space.

5. Conscious attention seems more parallel than this allows.

Attention is a different animal altogether. And I agree the system will be more complex than just a single 500 neuron space. I repeatedly said there might be more, and now I think there are almost certainly more. I think there may be one for every general area of the cortex because that seems to match the physiology/connectivity of the thalamus. And then there’s also short-term memory, etc. And then there’s those grid areas that allow orientation in space and time. (BTW, where are those grids?). All these are coordinated. But my working hypothesis is that what shows up in the “500 neuron space[s]” is what shows up in autobiographical consciousness.

6. Doesn’t account for glial cells and other potentially contributing systems in the brain.

[donning snide hat] [changing mind][doffing snide hat]

There are lots and lots of things that will have effects. Blood glucose level, etc. The question is what is minimally necessary to explain the effects we care about, i.e., what we are referring to when we refer to conscious experience. Glial cells may have a role, but if we can explain the basics with just neurons, that’s where we start.

*

• Wyrd Smythe

“Attention is a different animal altogether.”

We’ve been referring to the “spotlight of attention” and the “inner screen” all along. That’s all I mean. That’s all I’ve referred to — that inner screen.

“(BTW, where are those grid [cell]s?)”

(Hippocampus.)

“But my working hypothesis is that what shows up in the ‘500 neuron space[s]’ is what shows up in autobiographical consciousness.”

Yes, understood.

“There are lots and lots of things that will have effects.”

Right. Which have significant impact on the model.

“[I]f we can explain the basics with just neurons…”

So this is what’s called a “Toy” Model? (A simplified model of reality, typically created to study a single phenomenon without too many real-world distractions.)

• Wyrd Smythe

“You really (Really, REALLY) need to read this Introduction to the Semantic Pointer Architecture. Then get back to me.”

Okay, been there, done that.

As I just said to Mike, and as I’ve always said to you, the basic idea is a fine way to model mental concepts.

The page you pointed me to is part of an online Python package user documentation. That package apparently supports experimenting with semantic models.

From what I can tell, the unfortunately named SemanticPointer is the main attraction of the package. (I say “unfortunately named” because I see the conflation of two kinds of “pointers” as misleading. A better name is SemanticVector.)

This “pointer” appears to be a list of strings (the data parameter) which can be manipulated per a vocabulary (the vocab parameter) or an algebra.

As such, it’s kind of a made up game that allows you to make toy models that allow you experiment with semantic spaces.

Have you been using Python to actually implement any of the things you’ve talked about? I’d be curious to see what results you’re getting.

Bottom line, the vector space part of our conversation is what I always got a sense it was from you. I’m 100% fine with the idea of semantic spaces. I do think the idea of basis vectors is crucial, and I’d be intrigued to see how nengo_spa handles that. (And how it implements the three operations.)

I may take more of a peek at it when I get a chance, but for now, been there, done that, got the idea pretty good.

What I don’t see is any connection to neurons firing, so that’s one place to pick up the conversation.

• Wyrd Smythe

“For philosophical reasons, everything that happens can be described with an IPO model.”

Absolutely. Just keep in mind it’s a philosophical description, not reality.

“A process which meets the minimal constraints can be called a psychule.”

When I searched for “psychule” most of the hits come from your blog and from a bicycle club. This seems to be a pet theory of yours, not particularly supported by any literature in the field?

“What would be the role of an inner screen in the IPO model? It would be Input.”

You mean it’s an input to our momentary consciousness, right? The screen itself is an IPO module, of course. Our spotlight of attention receives its output as input?

“The Semantic Pointer Architecture explains how that screen can be implemented using a smallish set of neurons.”

I’ve shown why I think that’s not nearly enough, but granting an “inner screen” exists, your premise seems to be there is a (small) staging area for the focus of our mental content.

If I follow, you have an architecture of sub-conscious processes converging on a single (small) staging area that the conscious process “watches.” In the model I have, the spotlight moves around; there is no staging area.

“These operations can be performed using bio-plausible neurons as input.”

That’s the problem; I don’t think they can. Look at this page from the link you gave me. Pointers are created with semantics in the first place, not neurons firing.

The reason semantic vectors work is they’re entirely based on semantics.

Consider again the example of a Memory Space based on RAM bits (highly equivalent to Neuron Space based on neurons). Given two valid street address “vectors” in that space, you cannot create a valid new address by combining two valid ones. You get gibberish.

(It might be fun to code up an example to show you exactly what happens. Something to do while watching the ballgame tonight, maybe.)

“But what Eliasmith has done is built a neural-network model called SPAUN, implementing this functionality.”

I took a look at SPAUN. It “consists of 2.5 million simulated neurons,” and “can recognize numbers, remember them, figure out numeric sequences, and even write them down with a robotic arm.”

Here’s how well it works…

…with 2.5 million neurons.

The page you pointed to says, “In chapter 7 of How to build a brain, the SPA Unified Network (Spaun) model is presented that demonstrates how a wide variety of (about 8) cognitive tasks can be integrated in a single large-scale, spiking neuron model.”

And the architecture there encompasses everything from visual input to motor output.

That models like this can have success is well-known now. Deep learning neural networks use something kind of analogous.

But these things required millions of neurons.

• JamesOfSeattle

Yes, I coined psychule. But so far, I think it is supported by all kinds of literature in the field, from Kant, Husserl, Heidegger, Koch, Tononi, Russell, to Deutsch, Minsky, Dennett, Barr, etc. I think what you meant to say is that it has no recognition in the field. I’m just getting started. And since it is a side gig, and writing is clearly not my thing, it may take a while.

The screen itself is an IPO module, of course. Our spotlight of attention receives its output as input?

I think spotlight is a bad metaphor for attention. I think filter is better. By my model, attention controls what gets onto the screen. Attention controls which vector gets represented by the neurons. Conscious processes then follow.

If I follow, you have an architecture of sub-conscious processes converging on a single (small) staging area that the conscious process “watches.”

Better said “[I] have an architecture of sub-conscious processes converging on one of a few small staging areas which the conscious-type processes watch.” [Note plural of processes]

In the model I have, the spotlight moves around; there is no staging area.

Do you have a sketch as to how that would work, anatomically?

That’s the problem; I don’t think [vector operations] can ‘be performed using bio-plausible neurons as input

You’ll have to take that up with Chris Eliasmith. SPAUN uses standard-ish neural nets to recognize numbers from images, but the output of those neurons becomes the input to other neurons. Everything in SPAUN is neurons dealing with images and outputting through (simulated) muscle behavior.

Here’s how well [SPAUN recognizing numbers and drawing them] works…

Are you dissing SPAUN’s hand writing? You’ll note that SPAUN does not have a cerebellum. Also note that SPAUN is not copying what it sees. For example, look at the fours.

That models like this can have success is well-known now. Deep learning neural networks use something kind of analogous.

This is the only model I know of that can do multiple tasks, switching between them as appropriate, using only visual input and physical output, and using bio-plausible neurons.

*

• Wyrd Smythe

“I think it is supported by all kinds of literature in the field, from Kant, Husserl, Heidegger, Koch, Tononi, Russell, to Deutsch, Minsky, Dennett, Barr, etc.”

I’m pretty sure Kant never said anything about any kind of IPO module being conscious, and I’m dubious about some of the other names. Some of the latter ones don’t surprise me, but rather throwing names at me, how about arguing the point.

What makes a thermostat conscious? (I’m happy to read a paper if you point me to it, but names without citations don’t mean much.)

“Note plural of processes”

Yes, I haven’t forgotten. So does the spotlight of our focus move among these staging areas?

Why can’t it just move among the space of mental concepts? Multiple staging areas seems to weaken the idea of having a staging area in the first place.

“SPAUN uses standard-ish neural nets to recognize numbers from images, but the output of those neurons becomes the input to other neurons.”

I’d have to look at his model to see how he’s using vector operations. I don’t at all dispute interesting things are possible with various forms of neural network. I just don’t see any connection yet between this SPAUN and your inner screen or how you’re using vectors.

You haven’t yet connected any dots between SPAUN and your ideas. (Or I’ve missed them.)

“Are you dissing SPAUN’s hand writing?”

No. I’m pointing out that it takes 2.5 million neurons to accomplish as much as it does.

That should tell you something about the idea of using 500.

“This is the only model I know of…”

But, of course, reality is not bound by what we know.

• JamesOfSeattle

Also, here’s a paper purporting to explain the neural implementation: http://compneuro.uwaterloo.ca/files/publications/crawford.2015.pdf

• JamesOfSeattle

And before you say “but they don’t use neurons to generate the input”, note this paragraph from the paper just linked to:

The tasks of moving the output into the input for hierarchical traversals, controlling which vector is used as input to the query population, etc., are not neurally implemented here as they are peripheral to our central concern of representing human-scale structured knowledge in a biologically plausible manner. However, Spaun, a large scale, functional brain model constructed using the NEF, is evidence that it is possible to achieve this kind of control in a scalable spiking neural network (Eliasmith et al., 2012).

*

• Wyrd Smythe

And you think that proves…?

Please take a look at this post, Adventures in Address Vector Space, where I tried to illustrate the problem of semantics driven by low-level inputs.

A quick glance at the Crawford, Gingerich, Eliasmith paper suggests they’re doing something much more complicated.

• Wyrd Smythe

It’s a pretty dense 40-page paper but I’ll take a look when I can.

• Wyrd Smythe

Okay, as I mentioned below, I looked at the paper, and it seems to confirm a lot of what I thought, at least about semantic vectors themselves. Please see if my understanding matches yours.

So question: Is your vector generated by neurons in the screen or outside in the brain at large and sent to the screen? (I think you’re saying the latter?)

But either way the screen is, essentially, this vector sweeping around? How is it decoded? How does our consciousness make sense of this vector?

Okay, so I guess it’s two questions then: How is the vector encoded? How is it decoded?

• Wyrd Smythe

While the Twins were shutting out Detroit, I read some of that paper, Biologically Plausible, Human-Scale Knowledge Representation, by Crawford, Gingerich, and Eliasmith. Now I can respond a bit more about semantic vectors and how they are used.

Remember how I’ve been saying the basis vectors need to be semantic? I’ve been wondering how that was even possible. They way it’s done is kinda clever. The idea is an interesting one!

A hypersphere is an n-sphere with lots more dimensions than three. (The use of a sphere, rather than just an n-dimensional space, is so we have the idea of a unit sphere — the sphere with radius 1.)

So imagine a hypersphere with a lot of dimensions. Like 512 (which is what they used in the paper). In concrete terms, we’re talking about an array of 512 real numbers. Imagine that the numbers in the array are normalized (a common operation) such that:

$\sqrt{\sum_i^D x_i^2} = 1$

That gives us a vector that lies on the unit hypersphere.

A semantic vector is such a 512-number vector with randomly generated component numbers. The vector has a semantic tag (or label), such as {baseball} or {round} or {old}. To be meaningful, opposing concepts, different balls, different shapes, different states, are also added to the space.

In the paper, they added an entire English lexicon to the space. All the words.

They also created basis vectors (for that’s what all these are) for different kinds of relations between objects. For example, vectors for {baseball ISA ball} and {baseball IS round}.

Now, remember, these are all random unit vectors. A huge (tens of thousands) forest of vectors, but since each has 512 random numbers, the odds of overlap are essentially zero.

Once you have this semantic space, each of those vectors is a basis vector. Now you can create new vectors by combining existing vectors, because you start with semantic vectors in the first place.

The actual geometry of the space doesn’t really matter. Similar basis vectors (say {green} and {red}) can be far apart spatially, and similar ones can be very close. All that matters is relative operations between vectors. (I do wonder if “sorting” the space for like concepts would change things.)

What’s interesting is that you can combine vectors, say {baseball}*{old}*{mine}*{cheap}, to create a new vector. Imagine, also, a second vector, say {ipad}*{new}*{mine}*{valuable}.

Those could be combined, which would superpose them such that you could then use a vector that represented {new} to extract {ipad} from that combination. In a sense, it’s a kind of Fourier deal. You can mix lots of “sine waves” into a complex new one and then later extract the component sine waves.

Semantic vectors are a similar deal, but in large-dimensional space.

As a way of encoding semantics, it’s intriguing and looks like fun.

• JamesOfSeattle

So those are the vectors I’m talking about, implemented in neurons as described. I’m not sure where that leaves us. What does that change for you?

*

• Wyrd Smythe

Up above I asked you two questions. I’ll ask them again here:

1. Is your vector generated by neurons in the screen or outside in the brain at large and sent to the screen? (I think you’re saying the latter?)

2. But either way the screen is, essentially, this vector sweeping around? How is it decoded? How does our consciousness make sense of this vector?

That is: How is the vector encoded? How is it decoded?

• JamesOfSeattle

[this is, of course, speculation, but I am aiming for plausability]

1. Neurons outside the “screen” impinge on the screen to the effect that after they are done, the screen is in a state that it is representing the vector. So, yes, the latter.

2. Not sure it needs to be “decoded” other than by the methods described by Eliasmith. So to use SPAUN, a set of neurons outside the space recognizes a 4 in an image and generates a “four” in the appropriate vector space. Some neural mechanism “decodes” that “four” into a series of actions which draws a numeral 4.

Outside of Eliasmith’s model, and under the proper influence of other outside neurons, I expect the vector-representing neurons may also reactivate some subset of the cortex neurons responsible for that vector in the first place.

I have not studied Eliasmith’s stuff in depth so I cannot provide details. What issues do you see?

*

• Wyrd Smythe

“Some neural mechanism “decodes” that “four” into a series of actions which draws a numeral 4.”

The paper you pointed me to didn’t have any info about the numeral detection and replication. The results there entirely involve semantic vectors. Do you have a link with details to the vision and motor stuff? I’d like to know more about the architecture of that system.

“What issues do you see?”

Well, let’s have some examples for definiteness. Let’s talk about four general classes of very different mental concepts I have: Baseball, Jennifer Aniston, Science Fiction, and Dogs. (B, JA, SF, D)

In the four classes, there are many distinct concepts, many different thoughts about B, JA, SF, and D. (I can talk about baseball or science fiction for days!)

Presumably, my neural net has all these concepts embedded. I assume they are separate? I can, by effort of will, bring any of these concepts to focus. (Call the groups of neurons involved BN, JAN, SFN, and DN.)

So let’s consider the inner screen as the [P] in an IPO system. The [I] is whatever drives, feeds, or controls, the screen. The [O] is whatever the screen drives, or outputs, or which is “observed” on the screen.

So do the BN, JAN, SFN, and DN, groups, which are on the [I] side, generate their own vectors that are sent to the screen, or do they somehow work in concert?

If the BN group is feeding concepts of baseball to the screen, how does the JAN (or other) group take over and feed them?

What exactly are they feeding the 500-neuron group? Do all [I] neuron groups converge on the 500?

So then, on the [O] side, those 500 neurons stimulate some possibly large group of neurons. Neurons have, on average, 7000 connections, so if your thalamus neurons are average, 500×7000 = 5,300,000 possible neurons on the output side?

Are you suggesting all the thought machinery on the [O] side is completely general? It handles baseball, Jennifer Aniston, science fiction, dogs, mathematics, blogging, and everything else?

• JamesOfSeattle

I’m guessing, based on what (very) little I know, that it might work like the following:

Thinking of the screen as P is exactly wrong. Better to think of the screen as O for the attention process, and I (input) for the conscious process.

BN, JAN, SFN, and DN are constantly firing (in the cortex) at least at a low level. Apparently BN and SFN are usually firing at a higher level than the others, but at the moment they are each boosted because you are reading this. For the purpose of what gets to the screen[s], you could consider their firing as competition.

Some number of mechanisms will boost some N’s and inhibit other N’s. Taken altogether, these mechanisms can be considered the P for which BN, JAN, etc. are inputs and whatever ends up on screens (as vectors) are outputs. We can call this P the attention mechanism. All I groups converge on one or more screen. (The I groups may also go to other places in addition.)

Suppose one Input group is chosen via the attention mechanism, and maybe that choice is realized by synchronizing that group at 40hz. Perhaps a group firing at 40hz induces the neurons in the screen to fire in the pattern of a particular vector. When a different group is promoted to 40hz, that group influences the screen neurons to start firing in a different pattern, representing a different vector.

Regarding, the output from the screen, I should point out that many of those 5,300,000 connections are internal, going to other neurons within the 500. As far as I know, it could be most, but I don’t know. But in any case, some of those connections may be going to the amygdala, some to the PFC, some to the hippocampus, etc. Very possibly they could be going to those destinations in groups of 500, in which case it would be like communicating the same vector to each one of those places. (Thus, broadcasting, although not necessarily globally.). These connections, which are the outputs of the attention mechanism, would be the inputs of multiple IPO’s, i.e., multiple mechanisms. [These other IPO’s would be the psychules of the autobiographical self.]

So I’m guessing the amygdyla would be more likely to respond strongly to the JAN and DN than it would to the BN or SFN, depending on your emotional response to SciFi and baseball in general, but I don’t know. I expect memory of various kinds might work via the connections to the hippocampus. I think maybe specific actions, like going to the tv and finding the channel playing baseball may be the result of connections to the basal ganglia, possibly in coordination with PFC.

*
[working on a paper ref.]

• Wyrd Smythe

“Better to think of the screen as O for the attention process, and I (input) for the conscious process.”

Which is exactly what I suggested. (You’re not seeing that the screen has an [O] to the [I] of consciousness, and the screen has an [I] for the [O] of the attention process.) I was dividing things into “before” the screen and “after” the screen.

“…and whatever ends up on screens (as vectors) are outputs.”

So we have all these groups, BN, JAN, SFN, DN, and many, many more creating their “vectors” and competing to have them displayed on the screen.

What makes a “vector” from one group make sense relative to the “vector” from another group? Why do they speak the same “vector” language?

And what is that language? How exactly does BN send one of the zillions of different baseball “vectors” to the screen? How is that vector coded?

What happens if a BN “vector” happens to look like a SFN “vector” — why isn’t my consciousness confused?

“Regarding, the output from the screen, I should point out that many of those 5,300,000 connections are internal, going to other neurons within the 500.”

That would seem to limit the ability of the “signal” from the screen to get out of the 500 group.

You go on to say connections could go various places. Are you saying the amygdala, PFC, and hippocampus, all use the same vector language?

All the more crucial to explain what the protocol of that language actually is. How does one region of the brain communicate a vector to another?

(I continue to think we’re just talking about neuron signals and this vector business is just a way of describing things.)

• JamesOfSeattle

Regarding the role of the screen in IPO, I missed that you were including the neurons of the screen in the Attention Process. That’s fine.

”So we have all these groups, BN, JAN, SFN, DN, and many, many more creating their “vectors” and competing to have them displayed on the screen.”

The neural groups (BN, JAN, etc.) do not necessarily create the vectors separate from the screen. The vectors could exist only on the screen. To give a low-res example, say the screen is 6 binary numbers, so the zero vector is ‘0,0,0,0,0,0’. Suppose the vector for baseball is 0,0,1,0,0,0. BN would simply influence the screen until the screen is firing in the 0,0,1,0,0,0. BN does not have to be firing in a vector pattern. For example, it’s possible that during a change, some process sets all the bits to 0, then all BN has to do is flip the 3rd bit.

There does not need to be a vector language as long as each vector is reasonably unique. But as you say above, “remember, these are all random unit vectors. A huge (tens of thousands) forest of vectors, but since each has 512 random numbers, the odds of overlap are essentially zero.”

“What happens if a BN “vector” happens to look like a SFN “vector” — why isn’t my consciousness confused[?]”

There are error correction mechanisms available, but even so, mistakes are made. Sometimes your consciousness is confused.

And you are correct, we are just talking about neuron signals, but sometimes neuron signals are acting in concert, and vector is just a shorthand way to talk about that. Actually, the main reason we talk about vectors at all is the stuff above about vector operations, binding and unbinding especially.

*

• Wyrd Smythe

Sorry for the delay. I’ve been working on posts (and watching the Twins).

BTW: The posts upstream from this may (or may not) interest you. I’m on a one-a-day blogging mission to go over and discuss various consciousness topics. Many of the posts involve reactions, one way or another, to conversations I’ve had here or on Mike’s blog, so FWIW.

“There does not need to be a vector language as long as each vector is reasonably unique.”

What I meant is how one group of neurons communicates a given vector to another group. Per what you wrote, apparently BN, JAN, SFN, DN, and all the other N groups, have sufficient control over the 500 screen neurons to control the firing of all 500.

That means, ultimately, since there are a lot of N groups — millions, I should think — then a lot of cortex neuron groups have to be each sending enough signals to influence the screen.

“‘A huge (tens of thousands) forest of vectors, but since each has 512 random numbers, the odds of overlap are essentially zero.'”

Yes, but the random numbers in question are real numbers. They have essentially infinite degrees of freedom. The neuron vectors you’re talking about have only five.

And the numbers aren’t random. You’ve tied them to neuron firing, so the vectors represented by the 500 neurons in the screen firing have neuron semantics — they represent what the neurons are physically doing.

Semantic vectors (as described in the paper you linked to) aren’t anything like that. They’re arrays (vectors) of 512 real random numbers — 512 floating-point numbers!

(That paper only discussed their work with semantics, word connections and such. I’d like to see the architecture that connects semantic vectors, or any vectors, with actual neurons. In the paper they mention G, the neuron’s activation function, but don’t go much beyond that. Unless it’s in the really mathematical parts I haven’t gone over, yet.)

Not to the extent of thinking science fiction is baseball!

My concern is you have millions of N groups creating “vectors” (neuron firings) for one target, and this requires those “vectors” be significantly different so I distinguish baseball, Jennifer Aniston, science fiction, dogs, driving a car, mathematics, music, and so much, as truly distinct.

That seems asking a lot. It almost seems to require all those N groups somehow agree which part of the “vector” space they’ll use.

And, again, a 500-neuron inner screen just can’t be right (for reasons I’ve explained).

But, hey, whatever. One thing I’m always right about: I could be wrong.

• Wyrd Smythe

I found the idea of semantic vectors intriguing, so once I understood the three operations associated with them, I took a stab at a Python class implementing them.

Really interesting results. See my above comment for how they work.

It’s important to know about the dot product for vectors. Basically it’s a measure of how far apart two vectors are. It’s a single number that encodes the angle of the two vectors and their magnitudes.

On the unit (hyper)sphere, identical vectors have a dot product of +1.0. If they point in exactly opposite directions, the dot product is -1.0. Otherwise the value varies with 0.0 indicating they are perpendicular to each other. (Or that one of the vectors is all zeros.)

So dot product is a measure of similarity between vectors with +1.0 being identical.

So the first thing that’s interesting is what happens if you create a bunch of vectors:

+1.00000000: North
+0.05466264: cat
+0.03586380: horse
+0.03196142: play
+0.00539868: green
+0.00468251: blue
+0.00046639: mouse
-0.00023965: yellow
-0.01372053: red
-0.01743405: dog
-0.03875351: hold
-0.07799605: ride
-1.00000000: South

The above are a bunch semantic vectors I created. All have random values, except I constructed the South vector to be opposite of the North vector. (I wanted to verify and test the dot product math.)

The numbers are the dot products between the vector and the North vector. As expected, when North is compared to North, the dot product is +1.0, and when South is compared to North, the dot product is -1.0.

All the others, regardless of the random direction they point in 512-dimension space, read as being more or less perpendicular to North. Which is exactly what we’d hope for. (It’s all those dimensions that does it.)

Here’s where the rubber met the road:

v1 = dog.bind(play)
v2 = play.unbind()
v3 = v1.bind(v2)
vx = find_closest(v3)

Which gave me:

dog:play:yalp: {+0.93518921:dog}

The name gets constructed from the parts used. Unbinding uses a reverse name. So dog*play results in a new vector named “dog:play” (which is just a name, has nothing to do with the vector), and the vector created by unbinding that gives the name “dog:play:yalp” — the last reversed “play” being added.

More to the point, the resulting vector was very close to the dog vector showing that the unbind found the dog component in dog:play. (The find_closest function searches the vectors for the closest match.

It’s kind of like adding different sine wave to create a complex sound, and then doing Fourier Analysis on the sound to extract the sine waves.