BGP uses a simple finite state machine (FSM). As a result, BGP is always in one of the following 6 states: idle, connect, active, opensent, openconfirm and established. The RFC offers a full description of the FSM and exactly what events can lead to what state.
A summary of these states can be described as follows:
This is the initial state. In this state, all inbound BGP connections are refused and no resources are allocated to the peer. BGP is awaiting a start event.
In this state, the BGP speaking router is waiting for the TCP connection to be completed. If the TCP connection is successfully established, the BGP speaking router sends an Open message and transitions to the OpenSent state. If the TCP connection fails , the BGP speaking router move to the Active state.
The BGP speaking router was unable to successfully establish a TCP connection. In this state, the FSM will try to acquire a peer. Not by initiating a TCP connection, but by listening for and accepting one. If the TCP connection is successful, the BGP router will send an Open message to its peer and transition to the OpenSent state. If the TCP connection is unsuccessful, the FSM move (back) to Idle.
In this state, the BGP FSM is waiting for an Open Message from its peer. When an Open message is received, the message is checked for errors. When an error is found, the FSM transitions to Idle and the BGP speaking routers sends a Notification to its peer. During this state, the BGP speaking router can also receive a Notification message from its peer. This is in case that peer detected an error in the Open message. When this happens, the FSM also transitions to the status Idle. When there is no error detected inside the Open message, the BGP speaking router sends a Keepalive message and the FSM transitions to the OpenConfirm state.
In this state, the BGP speaking router is waiting for a Keepalive or Notification message. The peer will send a Notification if something wrong is detected with the Open message. Alternatively, the peer will send a Keepalive when the Open message is analyzed and no errors were found. When a Notification message is received, the FSM transitions to the Idle status. When a Keepalive is received, the FSM transitions to the status Established.
When the FSM is in the Established state, the BGP speaking routers can exchange Update, Notification and Keepalive messages. If and when a Notification message is received in this state, the BGP FSM transitions to the Idle state. The same thing happens when the negotiated hold time expires before a Keepalive is received from the peer.