Build fails to download artifact reporting a timeout
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
A build in a remote agent will fail the moment when the artifact download happens, and the agent logs will report a timeout.
Environment
Bamboo 6 or later.
Diagnosis
The build will fail and in the build log we'll see:
build log
1
error 27-Jul-2021 22:24:34 Unable to download artifact Non required shared artifact: [My_Artifact], pattern: [**] anchored at: [path]The agent log will report the following error:
atlassian-bamboo-agent.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
INFO | jvm 20 | 2021/07/27 22:23:14 | 2021-07-27 22:23:13,990 INFO [0-BAM::agent1::Agent:pool-3-thread-1] [TaskExecutorImpl] Starting task 'MyTask' of type 'com.atlassian.bamboo.plugins.bamboo-artifact-downloader-plugin:artifactdownloadertask'
INFO | jvm 20 | 2021/07/27 22:24:34 | 2021-07-27 22:24:34,066 ERROR [0-BAM::agent1::Agent:pool-3-thread-1] [BambooArtifactHttpTransport]
INFO | jvm 20 | 2021/07/27 22:24:34 | org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:263)
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183)
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
INFO | jvm 20 | 2021/07/27 22:24:34 | at java.util.zip.InflaterInputStream.fill(Unknown Source)
INFO | jvm 20 | 2021/07/27 22:24:34 | at java.util.zip.InflaterInputStream.read(Unknown Source)
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:656)
INFO | jvm 20 | 2021/07/27 22:24:34 | at java.io.InputStream.read(Unknown Source)
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2314)
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.commons.io.IOUtils.copy(IOUtils.java:2270)
INFO | jvm 20 | 2021/07/27 22:24:34 | at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2291)
INFO | jvm 20 | 2021/07/27 22:24:34 | at com.atlassian.bamboo.v2.build.agent.messages.ArtifactStreams.deserialiseStreamToDir(ArtifactStreams.java:89)
INFO | jvm 20 | 2021/07/27 22:24:34 | at com.atlassian.bamboo.build.artifact.BambooArtifactHttpTransport.retrieve(BambooArtifactHttpTransport.java:227)
INFO | jvm 20 | 2021/07/27 22:24:34 | at com.atlassian.bamboo.build.artifact.BambooRemoteArtifactHandler.lambda$retrieve$3(BambooRemoteArtifactHandler.java:83)
INFO | jvm 20 | 2021/07/27 22:24:34 | at com.atlassian.bamboo.executor.RetryingTaskExecutor.rerun(RetryingTaskExecutor.java:112)
INFO | jvm 20 | 2021/07/27 22:24:34 | at com.atlassian.bamboo.executor.RetryingTaskExecutor.runTask(RetryingTaskExecutor.java:99)
INFO | jvm 20 | 2021/07/27 22:24:34 | at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:207)On the server side, we will see:
atlassian-bamboo.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
2021-07-27 22:23:50,160 WARN [http-nio-8085-exec-437] [ArtifactStreams] Error during artifact transfer, total bytes written: 313849913, total requested: 314492985, latest request: 3584000
org.apache.catalina.connector.ClientAbortException: java.net.SocketTimeoutException
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:372)
at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:841)
at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:746)
at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:407)
at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:385)
at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96)
at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:253)
at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:211)
(...)
Caused by: java.net.SocketTimeoutException
at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:134)
at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:157)
at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1311)
at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:716)
at org.apache.tomcat.util.net.SocketWrapperBase.writeBlocking(SocketWrapperBase.java:536)
at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:480)
at org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.doWrite(Http11OutputBuffer.java:623)
at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:121)
at org.apache.coyote.http11.Http11OutputBuffer.doWrite(Http11OutputBuffer.java:225)
at org.apache.coyote.Response.doWrite(Response.java:600)
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:360)
... 174 moreCause
In this article we see the agent closing the connection. The operation fails after 80 seconds, but this doesn't mean that the timeout is actually 80 seconds, but for sure it's less than that.
It's difficult to say why this happens, but a very common cause for network latency is network congestion. On some edge cases there can be hardware issues, but we will discard that path for now.
Ideally a network monitoring system will give more context, especially when it comes to the connection between agent and server. Bottlenecks can happen anywhere in between, so ideally it's needed to have in mind every node that is involved in the connection (proxies, firewalls, etc).
Solution
This article will focus on how to increase the timeout value on Bamboo Server.
⚠️ This might or might not be enough to avoid this problem. If it doesn't help, you will need to continue investigating your network, to see where the request times out.
To increase the timeout value, please do the following:
Stop Bamboo.
Open the file <bamboo_server_directory>/conf/server.xml.
Spot your Connector. It will look similar to this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Connector
port="8085"
protocol="HTTP/1.1"
maxThreads="150" minSpareThreads="25"
connectionTimeout="20000"
disableUploadTimeout="true"
acceptCount="100"
enableLookups="false"
maxHttpHeaderSize="8192"
useBodyEncodingForURI="true"
URIEncoding="UTF-8"
redirectPort="8443"
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
</Connector>Change the connectionTimeout value to a bigger one. For example: 60000.
Start Bamboo again.
Was this helpful?