Consider N neurons = 1, … , N with values X_i = +1, -1.

The update rule is applied to the node i is given by:

If h_i ≥ 0 then x_i → 1 otherwise x_i → -1

Where hi = is called field at i, with b£ R a bias.

Thus, xi → sgn(hi), where the value of sgn(r)=1, if r ≥ 0, and the value of sgn(r)=-1, if r < 0.

We need to put bi=0 so that it makes no difference in training the network with random patterns.

We, therefore, consider hi= .

Consider, K = {-1, 1} ^N so that each state x £ X is given by xi £ { -1,1 } for 1 ≤ I ≤ N

we get 2^N possible states or configurations of the network.

We can describe a metric X by using the Hamming distance between any two states:

P(x, y) = # {i: xi≠y_i}

N Here, P is a metric with 0≤H(x,y)≤ N. It is clearly symmetric and reflexive.

With any of the asynchronous or synchronous updating rules, we get a discrete-time dynamical system.

The updating rule up: X → X describes a map.

And Up: X → X is trivially continuous.

Suppose we have only two neurons: N = 2

There are two non-trivial choices for connectivities:

w₁₂ = _w21 = 1

w₁₂= _w21 = -1

Here, we need to update Xm to X'm and denote the new energy by E' and show that.

E'-E = (X_m-X'_m ) ∑i≠mWmiXi.

Using the above equation, if X_m = X_m' then we have E' = E

If X_m = -1 and X_m' = 1 , then X_m - X_m' = 2 and hm= ∑iWmiXi ? 0

Thus, E' - E ≤ 0

Similarly if X_m =1 and X_m'= -1 then X_m - X_m' = 2 and hm= ∑iWmiXi < 0

Thus, E - E' < 0.

Suppose the connection weight W_ij = W_ji between two neurons I and j.

If W_ij > 0, the updating rule implies:

If X_j = 1, the contribution of j in the weighted sum, i.e., W_ijX_j, is positive. Thus the value of X_i is pulled by j towards its value X_j= 1

If X_j= -1 then W_ijX_j , is negative, and X_i is again pulled by j towards its value X_j = -1

Thus, if W_ij > 0 , then the value of i is pulled by the value of j. By symmetry, the value of j is also pulled by the value of i.

If W_ij < 0, then the value of i is pushed away by the value of j.

It follows that for a particular set of values Xi ∈ { -1 , 1 } for;

1 ≤ i ≤ N, the selection of weights taken as _Wij = X_iX_j for;

1 ≤ i ≤ N correlates to the Hebbian rule.

Suppose the vector x→ = (x₁,…,x_i,…,x_N) ∈ {-1,1}N is a pattern that we like to store in the Hopfield network.

To build a Hopfield network that recognizes x→, we need to select connection weight W_ij accordingly.

If we select W_ij =ɳ X_iX_j for 1 ≤ i , j ≤ N (Here, i≠j), where ɳ > 0 is the learning rate, then the value of Xi will not change under updating condition as we illustrate below.

We have

It implies that the value of X_i, whether 1 or -1 will not change, so that x→ is a fixed point.

Note that - x→ also becomes a fixed point when we train the network with x→ validating that Hopfield networks are sign blind.