How to enable Client IP Forwarding For SSH Sessions by setting up Proxy protocol for Bitbucket Data Center.
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
To set up a clustered Bitbucket Data Center, you need a load balancer to distribute requests between nodes using the same base URL ( Install Bitbucket Data Center | Bitbucket Data Center 9.6 | Atlassian Documentation). The load balancer acts as a reverse proxy, replacing incoming request headers with its own.
For HTTPS (OSI model Layer 7), you can preserve client IP information by enabling the option forwardfor (or its equivalent), allowing Bitbucket to log both the original and proxy IPs.
For SSH, which operates at Layer 4 (Transport), there are no headers, so Bitbucket only sees the load balancer’s IP in its logs, making traffic analysis difficult.
The solution came with Proxy protocol, which adjusts the connection string from the load balancer to the application node in Bitbucket with additional information:
PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP + single space + PROXY_IP + single space + CLIENT_PORT + single space + PROXY_PORT + "\r\n"
#The following is an example of the proxy protocol line for IPv4
PROXY TCP4 198.51.100.22 203.0.113.7 35646 80\r\nYou can find more details on the PROXY protocol implementation in the following sources:
Exploring the PROXY Protocol blog post by Benjamin Boudreau
Environment
Bitbucket Data Center 7.20.x and newer
HAProxy 1.5-dev3 and newer
AWS Network Load balancer
Solution
Bitbucket part:
Bitbucket Data Center 7.20 and newer have PROXY protocol enabled by default.
However, you can always set it explicitly in $bitbucket_home/shared/bitbucket.properties:
plugin.ssh.haproxy.proxy-enabled=trueThat's the only requirement from the Bitbucket side. No further configuration is required. If that doesn't work, the issue originates from the load balancer itself.
Load balancer part - HAProxy scenario
To configure the PROXY protocol with the HAProxy 1.5-dev3 and newer, you need to enable the protocol support (send-proxy) for each cluster node in haproxy.cfg section for SSH backend:
backend bitbucket_ssh_backend
mode tcp
balance roundrobin
server bitbucket01 163.25.93.240:7999 check send-proxy port 7999
server bitbucket02 192.128.0.220:7999 check send-proxy port 7999
timeout server 15mRestart HAProxy after the changes and try the clone once again.
After the changes are successfully applied, you will see double IP addresses in access logs:
atlassian-bitbucket-access.log
CLIENT_IP,LOAD_BALANCER_IP | ssh | i@DX7BUKx631x44878x0 | username | 2022-05-25 10:31:38,622 | SSH - git-upload-pack '/demo/repo.git' | "SSH-2.0-OpenSSH_8.6" | - | - | - | - | - | 17i1kgv |
CLIENT_IP,LOAD_BALANCER_IP | ssh | o@DX7BUKx631x44878x0 | username | 2022-05-25 10:31:41,300 | SSH - git-upload-pack '/demo/repo.git' | "SSH-2.0-OpenSSH_8.6" | 0 | 727 | 22705297 | cache:hit, clone, protocol:2, ssh:user:id:2 | 2678 | 17i1kgv |Load balancer part - AWS Network Load balancer scenario
In the case of AWS, only Network Load Balancer supports PROXY protocol version 2.
To enable proxy protocol v2 using the new console:
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
On the navigation pane, under LOAD BALANCING, choose Target Groups.
Choose the name of the target group to open its details page.
On the Attributes tab, select Edit.
On the Edit attributes page, select Proxy protocol v2
Choose Save changes.
To enable proxy protocol v2 using the old console:
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
On the navigation pane, under LOAD BALANCING, choose Target Groups
Select the target group
Choose Description, Edit attributes.
For Proxy protocol v2, choose Enable.
Choose Save.
For AWS-CLI please use modify-target-group-attributes withp proxy_protocol_v2.enabled
For more up-to-date information about proxy protocol support for AWS NLB please check https://docs.aws.amazon.com/elasticloadbalancing/latest/network/elb-ng.pdf
For Citrix Netscaler, please check the following guide https://support.citrix.com/article/CTX224265/how-to-configure-netscaler-to-send-proxy-protocol-to-backend-servers (Atlassian doesn't provide any assistance with Netscaler. Please reach official Netscaler support).
Was this helpful?