How to enable debug logging for Bitbucket Mesh
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
When troubleshooting Bitbucket Mesh and/or Bitbucket Mesh Sidecar, we may need to increase the logging level and enable additional logging to capture the full interaction. Depending on the issue, we may want to increase the logging level of a specific class, capture all logging at a higher level, or update logging at startup. Because Mesh receives its configuration from Bitbucket, this page will describe how to update logging during startup and also while the application is running.
Environment
Bitbucket 8.x onwards
Solution
DEBUG logging at runtime
If we are experiencing an ongoing issue but the services are online, we can adjust the logging level on the fly by performing the following steps:
Runtime
1. Edit the bitbucket.properties file and add the properties
1
2
3
4
mesh.logging.logger.com.atlassian.bitbucket.mesh=DEBUG
mesh.logging.logger.com.atlassian.bitbucket.mesh.process.OsProcess=DEBUG
mesh.logging.logger.io.grpc=DEBUG
mesh.profiling.enabled=trueNote the prefix
When adding these properties to the bitbucket.properties file, we MUST include the 'mesh' prefix.
2. Use the Mesh Config Refresh endpoint
1
curl -iv -u $ADMIN:$ADMIN_PWD -X POST -H "Content-Type: application/json" $BASE_URL/rest/ui/admin/git/mesh/config/refreshWhen run, this endpoint will check bitbucket.properties for a list of properties and propagate changes to the Mesh Nodes (Remote and Sidecar) to take effect immediately. You can confirm if it worked by seeing the below in the atlassian-mesh.log file in <Bitbucket-home>/mesh/log directory:
1
2
3
INFO [main] - ROOT Switched to log level [DEBUG]
INFO [main] - c.a.bitbucket.mesh.process.OsProcess Switched to log level [DEBUG]
INFO [main] - io.grpc Switched to log level [WARN]Disable the Mesh debug
Follow the below steps to disable the Mesh debug and profiler logs after you're done troubleshooting:
Disable Mesh debug
1. Edit the bitbucket.properties file and add the properties:
1
2
3
4
mesh.logging.logger.com.atlassian.bitbucket.mesh=INFO
mesh.logging.logger.com.atlassian.bitbucket.mesh.process.OsProcess=INFO
mesh.logging.logger.io.grpc=INFO
mesh.profiling.enabled=false2. Use the Mesh Config Refresh endpoint to refresh and apply the above properties without restarting Bitbucket
1
curl -iv -u $ADMIN:$ADMIN_PWD -X POST -H "Content-Type: application/json" $BASE_URL/rest/ui/admin/git/mesh/config/refreshWhen run, this endpoint will check bitbucket.properties for a list of properties and propagate changes to the Mesh Nodes (Remote and Sidecar) to take effect immediately.
You can confirm if it worked by seeing the below in the atlassian-mesh.log file in <Bitbucket-home>/mesh/log directory:
1
2
3
INFO [grpc-server:thread-586] "ManagementService/SetConfiguration" (>1 <0) com.atlassian.bitbucket.mesh Switched to log level [INFO]
INFO [grpc-server:thread-586] "ManagementService/SetConfiguration" (>1 <0) io.grpc Switched to log level [INFO]
INFO [grpc-server:thread-586] "ManagementService/SetConfiguration" (>1 <0) c.a.b.m.c.DefaultProfilingConfig Profiling has been disabledDEBUG logging on start-up
If we are experiencing a failure during startup such as Mesh Sidecar failing to initialize, the updated logging configuration may never get sent or received. If this occurs, we can perform the following steps:
Start-up
1. Edit the mesh.properties file
Mesh Profile | Location | Example |
|---|---|---|
Mesh Sidecar |
|
|
Remote Mesh |
|
|
2. Add the logging properties and level
1
2
3
4
5
6
7
8
9
10
11
# The base Mesh logger
logging.logger.com.atlassian.bitbucket.mesh=DEBUG
# Mesh gRPC Logger
logging.logger.io.grpc=WARN
# Mesh Git Operations
logging.logger.com.atlassian.bitbucket.mesh.process.OsProcess=DEBUG
# Enable Mesh Profiler Logs
profiling.enabled=true3. Restart the JVM and check the logs
Within the logs, we should see messages indicating that the log level has been updated:
1
2
3
INFO [main] - ROOT Switched to log level [DEBUG]
INFO [main] - c.a.bitbucket.mesh.process.OsProcess Switched to log level [DEBUG]
INFO [main] - io.grpc Switched to log level [WARN]Was this helpful?