08 Nov 2021

#### BGP Improvements

For users of BGP you can now view the status of your BGP routers, including session status, RIB / FIB contents, and agent health via the new
`CalicoNodeStatus` API. See [the API documentation](https://docs.projectcalico.org/archive/v3.21/reference/resources/caliconodestatus){: data-proofer-ignore=""} for more
details.

In addition, you can control BGP advertisement of certain prefixes using the new `disableBGPExport` option on each IP pool, allowing greater control
of your route sharing scheme.

Pull requests:

 - Added Calico node status resource (`CalicoNodeStatus`) which represents a collection of status information for a node that Calico reports back to the user for use during troubleshooting. [libcalico-go #1502](https://github.com/projectcalico/libcalico-go/pull/1502) (@song-jiang)
 - Report node BGP status from calico/node. [node #1234](https://github.com/projectcalico/node/pull/1234) (@song-jiang)
 - Add new syncer for BGP status API. [typha #662](https://github.com/projectcalico/typha/pull/662) (@song-jiang)
 - Don't export BGP routes for IP pools that have disableBGPExport==true [confd #647](https://github.com/projectcalico/confd/pull/647) (@coutinhop)

#### Service-based network policy improvements

In v3.20, we introduced egress policy rules that can match on Kubernetes services. In v3.21, we improved upon that in two ways. First, you can now use
service matches in Calico NetworkPolicy and GlobalNetworkPolicy ingress rules. Second, you can now use service-based network policy rules on Windows
nodes.

Pull requests:

 - Policy ingress rules now support service selectors. [felix #3024](https://github.com/projectcalico/felix/pull/3024) (@mgleung)
 - Windows data plane support for Service-based network policy rules [felix #2917](https://github.com/projectcalico/felix/pull/2917) (@caseydavenport)
 - Allow services to be specified in the Source field of Ingress rules [libcalico-go #1517](https://github.com/projectcalico/libcalico-go/pull/1517) (@mgleung)

#### Option to run Calico as non-privileged and non-root

Calico can now optionally run in non-privileged and non-root mode, with some limitations. See the [documentation](https://docs.projectcalico.org/archive/v3.21/security/non-privileged){: data-proofer-ignore=""} for more information.

Pull requests:

 - Change node and supporting binary permissions so that they can be run as a non-root user [node #1224](https://github.com/projectcalico/node/pull/1224) (@mgleung)
 - CNI plugin now sets route_localnet=1 for container interfaces [cni-plugin #1168](https://github.com/projectcalico/cni-plugin/pull/1168) (@mgleung)
 - CNI plugins now have SUID bit set in order to run as non-root [cni-plugin #1168](https://github.com/projectcalico/cni-plugin/pull/1168) (@mgleung)

#### IPReservations API 

You can use the new [IPReservations API](https://docs.projectcalico.org/archive/v3.21/reference/resources/ipreservation){: data-proofer-ignore=""} to reserve certain IP addresses so that they will not be used by Calico IPAM. This allows for fine-grained
control of the IP space in your cluster.

 - Add support for IPReservations [libcalico-go #1509](https://github.com/projectcalico/libcalico-go/pull/1509) (@fasaxc)


#### Bug fixes

**General**

 - Downgrade repetitive info level logging in calico/node autodetection code [node #1237](https://github.com/projectcalico/node/pull/1237) (@caseydavenport)
 - Updated ubi base images and CentOS repos to stop CVE false positives from being reported. [node #1136](https://github.com/projectcalico/node/pull/1136) (@coutinhop)
 - Fixed typo in umount command [pod2daemon #64](https://github.com/projectcalico/pod2daemon/pull/64) (@ScheererJ)
 - Fixes this [bug](https://github.com/projectcalico/calico/issues/5018) which caused WireGuard stats to be collected even when WireGuard was disabled. Additionally, the version of the wgctrl dependency has been updated as the previous version caused thread leaks. [felix #3057](https://github.com/projectcalico/felix/pull/3057) (@mikestephen)
 - Fix blackhole route table interface matches to handle empty interface regexes. [felix #3007](https://github.com/projectcalico/felix/pull/3007) (@robbrockbank)
 - Fix slow performance when updating a Kubernetes namespace when there are many Pods (and in turn, slow startup performance when there are many namespaces). [felix #2964](https://github.com/projectcalico/felix/pull/2964) (@fasaxc)
 - Close race condition that could result in an extra IPAM block being allocated to a node. [libcalico-go #1488](https://github.com/projectcalico/libcalico-go/pull/1488) (@caseydavenport)
 - Fix that podIP annotation could be incorrectly clobbered for stateful set pods: https://github.com/projectcalico/calico/issues/4710 [libcalico-go #1472](https://github.com/projectcalico/libcalico-go/pull/1472) (@fasaxc)
 - Fix removal of old CNI configuration on name-change [cni-plugin #1153](https://github.com/projectcalico/cni-plugin/pull/1153) (@caseydavenport)
 - Readiness depends on all syncers [typha #613](https://github.com/projectcalico/typha/pull/613) (@robbrockbank)
 - Exclude RR nodes from BGP full mesh [confd #619](https://github.com/projectcalico/confd/pull/619) (@coutinhop)

**BPF dataplane**

 - Fixed a bug in ExternalTrafficPolicy=Local that lead to connection stalling. [felix #3015](https://github.com/projectcalico/felix/pull/3015) (@tomastigera)
 - Fixed broken connections when client used the same port to connect to the same backed via a nodeport on different nodes. [felix #2983](https://github.com/projectcalico/felix/pull/2983) (@tomastigera)
 - The eBPF mode implementation of DoNotTrack policy was incorrectly allowing an inbound connection through a HostEndpoint, when the HostEndpoint had DoNotTrack policy for the ingress direction but not for egress.  For precise compatibility with Calico's established DoNotTrack semantics, that connection should be disallowed, and now is.  (Because of the lack of connection tracking, successful use of DoNotTrack policy to allow flows requires configuring the DoNotTrack policy symmetrically in both directions.) [felix #2982](https://github.com/projectcalico/felix/pull/2982) (@neiljerram)

#### Other changes

**General**

 - calico/node logs write to /var/log/calico within the container by default, in addition to stdout [node #1133](https://github.com/projectcalico/node/pull/1133) (@song-jiang)
 - Read pod IP information from Amazon VPC CNI annotation, if present on the pod. [libcalico-go #1523](https://github.com/projectcalico/libcalico-go/pull/1523) (@caseydavenport)
 - Update etcd client version to v3.5.0 [libcalico-go #1495](https://github.com/projectcalico/libcalico-go/pull/1495) (@Aceralon)
 - Optimize lists and watches made against the Kubernetes API [libcalico-go #1484](https://github.com/projectcalico/libcalico-go/pull/1484) (@caseydavenport)
 - WorkloadEndpoints now support hostPorts [libcalico-go #1471](https://github.com/projectcalico/libcalico-go/pull/1471) (@AloysAugustin)
 - Include CNI plugin release v1.0.0 [cni-plugin #1141](https://github.com/projectcalico/cni-plugin/pull/1141) (@caseydavenport)
 - Allow configuration of num_queues for Calico created veth interfaces [cni-plugin #1116](https://github.com/projectcalico/cni-plugin/pull/1116) (@arikachen)
 - Typha now gives newly connected clients an extra grace period to catch up after sending the snapshot to reduce the possibility of cyclic disconnects. [typha #614](https://github.com/projectcalico/typha/pull/614) (@fasaxc)

**Windows** 

 - Windows VXLAN network adapter can be configurable when multi interfaces exist. [node #1166](https://github.com/projectcalico/node/pull/1166) (@knabben)
 - Windows: verify upgrade image using node pod [node #1323](https://github.com/projectcalico/node/pull/1323) (@lmm)
 - Add calico-node upgrade service for upgrades on Windows [node #1254](https://github.com/projectcalico/node/pull/1254) (@lmm)
 - Windows: fix install stuck at Wait-ForCalicoInit [node #1210](https://github.com/projectcalico/node/pull/1210) (@lmm)
 - Windows: replace WMI cmdlet usage with CIM [node #1156](https://github.com/projectcalico/node/pull/1156) (@lmm)
 - Windows: enable kube-proxy WinDSR flag for versions of 1809 that include DSR support [node #1015](https://github.com/projectcalico/node/pull/1015) (@lmm)

**Multi-arch**

 - eBPF arm64/aarch64 [node #1044](https://github.com/projectcalico/node/pull/1044) (@frozenprocess)

**BPF dataplane**

 - BPF: Endpoints in EndpointsSlices that are not ready are excluded from NAT [felix #3017](https://github.com/projectcalico/felix/pull/3017) (@tomastigera)
 - Calico's eBPF dataplane now fully implements [DoNotTrack policy](https://docs.projectcalico.org/security/high-connection-workloads) [felix #2910](https://github.com/projectcalico/felix/pull/2910) (@neiljerram)

**VPP dataplane** 

 - Add HostPort support in the gRPC dataplane [cni-plugin #1119](https://github.com/projectcalico/cni-plugin/pull/1119) (@AloysAugustin)
