Telepresence OSS Release Notes

The version and status commands will now output the image that the traffic-agent will be using when injected by the agent-injector.

The telepresence config view command will now print zero values in the output when the default for the value is non-zero.

The improvements made to be able to run the telepresence daemon in docker using telepresence connect --docker made it impossible to run both the CLI and the daemon in docker. This commit fixes that and also ensures that the user- and root-daemons are merged in this scenario when the container runs as root.

A telepresence intercept --replace did not correctly mount all volumes, because when the intercepted container was removed, its mounts were no longer visible to the agent-injector when it was subjected to a second invocation. The container is now kept in place, but with an image that just sleeps infinitely.

A telepresence intercept --replace will no longer switch the mode of the intercepted workload, forcing all subsequent intercepts on that workload to use --replace until the agent is uninstalled. Instead, --replace can be used interchangeably just like any other intercept flag.

The logic added to allow the root daemon to connect directly to the cluster using the user daemon as a proxy for exec type authentication in the kube-config, didn't take into account that a context name sometimes contains the colon ":" character. That character cannot be used in filenames on windows because it is the drive letter separator.

The AGENT_IMAGE was a concatenation of the agent's name and tag. This is now changed so that the env instead contains an AGENT_IMAGE_NAME and AGENT_INAGE_TAG. The AGENT_IMAGE is removed. Also, a new env REGISTRY is added, where the registry of the traffic- manager image is provided. The AGENT_REGISTRY is no longer required and will default to REGISTRY if not set.

Telepresence would sometimes prefix environment interpolation expressions in the traffic-agent twice so that an expression that looked like $(SOME_NAME) in the app-container, ended up as $(_TEL_APP_A__TEL_APP_A_SOME_NAME) in the corresponding expression in the traffic-agent.

A darwin machine with full access to the cluster's subnets will never create a TUN-device, and a check was missing if the device actually existed, which caused a panic in the root daemon.

The telepresence status and telepresence config view commands didn't show the allowConflictingSubnets CIDRs because the value wasn't propagated correctly to the CLI.

Only one host-based connection can exist because that connection will alter the DNS to reflect the namespace of the connection. but it's now possible to create additional connections using --docker while retaining the host-based connection.

The hostname of a containerized daemon defaults to be the container's ID in Docker. You now can override the hostname using telepresence connect --docker --hostname <a name>.

The output of the telepresence status when using --output json or --output yaml will either show an object where the user_daemon and root_daemon are top level elements, or when multiple connections are used, an object where a connections list contains objects with those daemons. The flag --multi-daemon will enforce the latter structure even when only one daemon is connected so that the output can be parsed consistently. The reason for keeping the former structure is to retain backward compatibility with existing parsers.

A quit (without -s) just disconnects the host user and root daemons but will quit a container based daemon. The message printed was simplified to remove some have/has is/are errors caused by the difference.

The agent-injector service will now refresh the secret used by the mutator-webhook each time a new connection is established, thus preventing the certificates to go out-of-sync when the secret is regenerated.

An intercept attempt that resulted in a timeout due to failure of injecting the traffic-agent left the telepresence-agents configmap in a state that indicated that an agent had been added, which caused problems for subsequent intercepts after the problem causing the first failure had been fixed.

A telepresence status, issued when multiple containerized daemons were active, would error with "multiple daemons are running, please select one using the --use <match> flag". This is now fixed so that the command instead reports the status of all running daemons.

A telepresence version, issued when multiple containerized daemons were active, would error with "multiple daemons are running, please select one using the --use <match> flag". This is now fixed so that the command instead reports the version of all running daemons.

A telepresence quit -s, issued when multiple containerized daemons were active, would error with "multiple daemons are running, please select one using the --use <match> flag". This is now fixed so that the command instead quits all daemons.

The DNS search path that Telepresence uses to simulate the DNS lookup functionality in the connected cluster namespace was not removed by a telepresence quit, resulting in connectivity problems from the workstation. Telepresence will now remove the entries that it has added to the search list when it quits.

The user-daemon would die with a fatal "fatal error: concurrent map writes" error in the connector.log, effectively killing the ongoing connection.

Intercepts didn't work when multiple service ports were using the same container port. Telepresence would think that one of the ports wasn't intercepted and therefore disable the intercept of the container port.

The root daemon would sometimes hang forever when attempting to disconnect due to a deadlock in the VIF-device.

The user daemon would panic if the traffic-manager was unreachable. It will now instead report a proper error to the client.

The telepresence helm installer will no longer discover and convert workloads that were modified by versions prior to 2.6.0. The traffic manager will and no longer support the muxed tunnels used in versions prior to 2.5.0.

This feature adds the following metrics to the Prometheus endpoint: connect_count, connect_active_status, intercept_count, and intercept_active_statusintercept_count metric has been renamed to active_intercept_count for clarity.

The docker image used when running a Telepresence intercept in docker mode can now be configured using the setting images.clientImage and will default first to the value of the environment TELEPRESENCE_CLIENT_IMAGE, and then to the value preset by the telepresence binary. This configuration setting is primarily intended for testing purposes.

The telepresence TUN-device is now capable of establishing direct port-forwards to a traffic-agent in the connected namespace. That port-forward is then used for all outbound traffic to the device, and also for all traffic that arrives from intercepted workloads. Getting rid of the extra hop via the traffic-manager improves performance and reduces the load on the traffic-manager. The feature can only be used if the client has Kubernetes port-forward permissions to the connected namespace. It can be disabled by setting cluster.agentPortForward to false in config.yml.

The root-daemon now communicates directly with the traffic-manager instead of routing all outbound traffic through the user-daemon. The root-daemon uses a patched kubeconfig where exec configurations to obtain credentials are dispatched to the user-daemon. This to ensure that all authentication plugins will execute in user-space. The old behavior of routing everything through the user-daemon can be restored by setting cluster.connectFromRootDaemon to false in config.yml.

telepresence connect (and other commands that kick off a connect) now accepts an --allow-conflicting-subnets CLI flag. This is equivalent to client.routing.allowConflictingSubnets in the helm chart, but can be specified at connect time. It will be appended to any configuration pushed from the traffic manager.

Print a warning if the minor version diff between the client and the traffic manager is greater than three.

The authenticator binary, used when serving proxied exec kubeconfig credential retrieval, has been removed. The functionality was instead added as a subcommand to the telepresence binary.

This flag is similar to --docker-build but will start the container with more relaxed security using the docker run flags --security-opt apparmor=unconfined --cap-add SYS_PTRACE.

In some situations it is necessary to make some ports available to the host from a containerized telepresence daemon. This commit adds a repeatable --expose <docker port exposure> flag to the connect command.

The kube-system and kube-node-lease namespaces should not be affected by a global agent-injector webhook by default. A default namespaceSelector was therefore added to the Helm Chart agentInjector.webhook that contains a NotIn preventing those namespaces from being selected.

Users of Telepresence < 2.9.0 that make use of the pod template TLS annotations were unable to upgrade because the annotation names have changed (now prefixed by "telepresence."), and the environment expansion of the annotation values was dropped. This fix restores support for the old names (while retaining the new ones) and the environment expansion.

Built Telepresence with go 1.21.3 to address CVEs.

When listing intercepts (typically by calling telepresence list) selectors of services are matched against workloads. Previously the match was made against the labels of the workload, but now they are matched against the labels pod template of the workload. Since the service would actually be matched against pods this is more correct. The most common case when this makes a difference is that statefulsets now are listed when they should.

-> The telepresence helm install/upgrade/uninstall commands will no longer start the telepresence user daemon because there's no need to connect to the traffic-manager in order for them to execute.

-> A race condition would sometimes occur when a Telepresence TUN device was deleted and another created in rapid succession that caused the routing table to reference interfaces that no longer existed.

When using telepresence connect --docker, a lingering container could be present, causing errors like "The container name NN is already in use by container XX ...". When this happens, the connect logic will now give the container some time to stop and then call docker stop NN to stop it before retrying to start it.

Files in the Telepresence cache are accesses by multiple processes. The processes will now use advisory locks on the files to guarantee consistency.

The behavior changed so that a connected Telepresence client is bound to a namespace. The namespace can then not be changed unless the client disconnects and reconnects. A connection is also given a name. The default name is composed from <kube context name>-<namespace> but can be given explicitly when connecting using --name. The connection can optionally be identified using the option --use <name match> (only needed when docker is used and more than one connection is active).

The global flags --context and --docker will now be considered deprecated unless used with commands that accept the full set of Kubernetes flags (e.g. telepresence connect).

The --namespace flag is now deprecated for telepresence intercept command. The flag can instead be used with all commands that accept the full set of Kubernetes flags (e.g. telepresence connect).

-> The telepresence code-base still contained a lot of code that would modify workloads instead of relying on the mutating webhook installer when a traffic-manager version predating version 2.6.0 was discovered. This code has now been removed.

These commands can be used to check accessible namespaces and for automation.

A deprecation warning will be printed if a command other than telepresence connect causes an implicit connect to happen. Implicit connects will be removed in a future release.

Rebuild Telepresence with go 1.21.1 to address CVEs.

Openshift users reported that the traffic agent injection was failing due to a missing security context.

ASLR hardens binary sercurity against fixed memory attacks.

Updated the release workflow files in github actions to including building and publishing the client binaries for arm64 architecture.

If provided, the KUBECONFIG environment variable was passed to the kubeauth-foreground service as a parameter. However, since it didn't exist, the CLI was throwing an error when using telepresence connect --docker.

The telepresence list --output json-stream wasn't releasing the session's lock after being stopped, including with a telepresence quit. The user could be blocked as a result.

Replace deprecated info in the JSON output of the telepresence list command.

Upgrading the traffic-manager using telepresence helm upgrade would sometimes result in a helm error message executing "telepresence/templates/intercept-env-configmap.yaml" at <.Values.intercept.environment.excluded>: nil pointer evaluating interface {}.excluded"

The traffic manager admin was forced to set it manually during the chart installation.

A new agent.helm.httpIdleTimeout setting was added to the Helm chart that controls the proprietary Traffic agent's http idle timeout. The default of one hour, which in some situations would cause a lot of resource consuming and lingering connections, was changed to 70 seconds.

Several gauges were added to the Prometheus client to make it easier to monitor what the Traffic manager spends resources on.

Add option to set traffic agent pull policy in helm chart.

Fixes a resource leak in the Traffic manager caused by lingering tunnels between the clients and Traffic agents. The tunnels are now closed correctly when terminated from the side that created them.

Fixes a regression introduced in version 2.10.5, making it impossible to set the traffic-manager namespace using the telepresence.io kubeconfig extension.

The DNS configuration now supports two new fields, excludes and mappings. The excludes field allows you to exclude a given list of hostnames from resolution, while the mappings field can be used to resolve a hostname with another.

Added a new config map that can take an array of environment variables that will then be excluded from an intercept that retrieves the environment of a pod.

A traffic-agent of version 2.13.3 (or 1.13.15) would not propagate the directories under /var/run/secrets when used with a traffic manager older than 2.13.3.

A context cancellation could sometimes be trapped in a stream reader, causing it to incorrectly return an undefined message which in turn caused the parent reader to panic on a nil pointer reference.

Telepresence will now attempt to detect and report routing conflicts with other running VPN software on client machines. There is a new configuration flag that can be tweaked to allow certain CIDRs to be overridden by Telepresence.

Running telepresence test-vpn will now print a deprecation warning and exit. The command will be removed in a future release. Instead, please configure telepresence for your VPN's routes.

Add .Values.hooks.curl.imagePullSecrets and .Values.hooks curl.imagePullSecrets to Helm values.

The default setting of the reinvocationPolicy for the mutating webhook dealing with agent injections changed from Never to IfNeeded.

The eks.amazonaws.com/serviceaccount volume injected by EKS is now exported and remotely mounted during an intercept.

The mutating webhook now correctly applies the namespace selector even if the cluster version contains non-numeric characters. For example, it can now handle versions such as Major:"1", Minor:"22+".

The "telepresence" Docker network will now propagate DNS AAAA queries to the Telepresence DNS resolver when it runs in a Docker container.

Running telepresence intercept --local-only --docker-run no longer results in a panic.

Running telepresence intercept --local-only --mount false no longer results in an incorrect error message saying "a local-only intercept cannot have mounts".

The helm chart now correctly handles custom agentInjector.webhook.port that was not being set in hook URLs.

Params .intercept.disableGlobal and .timeouts.agentArrival are now correctly honored.

Replaced / characters with a - when the authenticator service creates the kubeconfig in the Telepresence cache.

Configurable strategy (auto, powershell. or registry) to set the global DNS search path on Windows. Default is auto which means try powershell first, and if it fails, fall back to registry.

The timeout for the traffic manager to wait for traffic agent to arrive can now be configured in the values.yaml file using timeouts.agentArrival. The default timeout is still 30 seconds.

The automatic discovery of a local container based cluster (minikube or kind) used when the Telepresence daemon runs in a container, now works on macOS and Windows, and with different profiles, ports, and cluster names

Multiple simultaneous intercepts can transfer large files in bidirectionally and in parallel.

Pods using persistent volumes no longer causes timeouts when intercepted.

Ensure that `telepresence connect`` succeeds even though DNS isn't configured correctly.

The traffic-manager would sometimes panic with a "close of closed channel" message and exit.

The traffic-manager would sometimes panic and exit after some time due to a type cast panic.

Improve login behavior by clearing the saved intermediary API Keys when a user logins to force Telepresence to generate new ones.

The malfunction of the Ambassador Telepresence Agent occurred as a result of an update which compressed the executable file.

The Docker network used by a Kind or Minikube (using the "docker" driver) installation, is automatically detected and connected to a Docker container running the Telepresence daemon.

Mapped namespaces are included in the output of the telepresence status command.

There's a new --address flag to the intercept command allowing users to set the target IP of the intercept.

The client will no longer need cluster wide permissions when connected to a namespace scoped Traffic Manager.

The Traffic Manager now uses a fail-proof way to determine the cluster domain.

DNS on windows is more reliable and performant.

The agent is now correctly injected even with a high number of deployment starting at the same time.

The kubeconfig is made self-contained before running Telepresence daemon in a Docker container.

The version command won't throw an error anymore if there is no kubeconfig file defined.

Update Golang to 1.20.3 to address CVE-2023-24534, CVE-2023-24536, CVE-2023-24537, and CVE-2023-24538

Telepresence now includes the kubeauth logs when running the gather-logs command

Telepresence now propagates environment variables properly to the kubeauth-foreground to be used with cluster authentication

Telepresence now enables you to check for a service and pod's connectivity independently, so that it can proxy one without proxying the other.

Authentication to EKS and GKE clusters have been fixed (k8s >= v1.26)

Telepresence will not longer panic when a CNAME does not contain the .svc in it

The multi-arch build for the ambassador-telepresence-manager and ambassador-telepresence-agent now works for both amd64 and arm64.

Some labels in the Helm chart for the Ambassador Agent were duplicated, causing problems for FluxCD.

The ambassador-telepresence-manager and ambassador-telepresence-agent are now distributed as multi-architecture images and can run natively on both linux/amd64 and linux/arm64.

The connectivity check failed to recognize that the connected peer wasn't a traffic-manager. Consequently, it didn't proxy the cluster because it incorrectly assumed that a successful connect meant cluster connectivity,

The telepresence test-vpn did not include routes of type link when checking for subnet conflicts.

Fixed a bug that prevented the local daemons from automatically reconnecting to the traffic manager when the network connection was lost.

Telepresence can now create intercepts with traffic-managers of version 2.9.5 and older.

Ensure that CLI and user-daemon binaries are the same version when running telepresence helm install or telepresence helm upgrade.

telepresence check-vpn no longer crashes when the daemons don't start properly.

There was an issue downloading the free enhanced client. This problem was fixed, 2.8.0 was reinstated

Rollback 2.8.0 while we investigate an issue with ambassador cloud.

The Telepresence DNS resolver is now capable of resolving queries of type A, AAAA, CNAME, MX, NS, PTR, SRV, and TXT.

A new client struct was added to the Helm chart. It contains a connectionTTL that controls how long the traffic manager will retain a client connection without seeing any sign of life from the client.

A dns element was added to the client struct in Helm chart. It contains an includeSuffixes and an excludeSuffixes value that controls what type of names that the DNS resolver in the client will delegate to the cluster.

The API port used by the traffic-manager is now configurable using the Helm chart value apiPort. The default port is 8081.

An new agent struct was added to the Helm chart. It contains an `envoy` structure where the server and admin port of the Envoy proxy running in the enhanced traffic-agent can be configured.

The Helm chart dnsConfig was deprecated but retained for backward compatibility. The fields alsoProxySubnets and neverProxySubnets can now be found under routing in the client struct.

The Helm chart agentInjector.agentImage was moved to agent.image. The old value is deprecated but retained for backward compatibility.

The Helm chart agentInjector.appProtocolStrategy was moved to agent.appProtocolStrategy. The old value is deprecated but retained for backward compatibility.

The Helm chart dnsServiceName, dnsServiceNamespace, and dnsServiceIP has been removed, because they are no longer needed. The TUN-device will use the traffic-manager pod-IP on platforms where it needs to dedicate an IP for its local resolver.

The former options `-u` and `-r` for `telepresence quit` has been deprecated and replaced with one option `-s` which will quit both the root daemon and the user daemon.

Environment variable interpolation now works for all definitions that are copied from pod containers into the injected traffic-agent container.

An attempt to create simultaneous intercepts that span multiple namespace on the same workstation is detected early and prohibited instead of resulting in failing DNS lookups later on.

Spurious and incorrect ""!! SRV xxx"" messages will no longer appear in the logs when the reason is normal context cancellation.

Single label names now resolves correctly when using Telepresence in Docker on a Linux host

The Helm chart value appProtocolStrategy is now correctly named (used to be appPortStategy)

The resources for the traffic-agent container and the optional init container can be specified in the Helm chart using the resources and initResource fields of the agentInjector.agentImage