Git LFS fails with HTTP error over SSH: failed to push some refs to and Client error 413
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
Pushing attempt with Git LFS results in when cloning, pushing or pulling/fetching, the client may fail like so:
1
2
3
4
5
6
$ git push origin master # using SSH
Git LFS: (186 of 187 files, 4 skipped) 268.25 MB / 448.34 MB, 17.66 MB skipped
Client error: https://your.bitbucket.com/rest/git-lfs/storage/proj/repo/84c72136e4d22c2c42819d902b4400a9cf73f2b04fd6f9f8666739c3bc0a0f79 from HTTP 413
Client error: https://your.bitbucket.com/rest/git-lfs/storage/proj/repo/1c9335681f70363edeb00820ef394fac66125856cca0c3c8cf2c66636739f277 from HTTP 413
Client error: https://your.bitbucket.com/rest/git-lfs/storage/proj/repo/e79f7abda0ba90f2be4e613f78d1ecb7344970711106fe80d0028106f97f613c from HTTP 413
error: failed to push some refs to 'ssh://git@your.bitbucket.com:7999/proj/repo.git'Cause
This indicates the repository does not have LFS enabled. A user with REPO_ADMIN permissions is able to enable LFS support.
Analysis
The following come up on different logs:
atlassian-bitbucket.log
1
2016-02-02 14:03:12,626 INFO [NioProcessor-8] c.a.b.i.s.s.SelfClosingChannelSession Executing command: git-receive-pack '/proj/repo.git' atlassian-bitbucket-access.log
1
192.168.1.10 | ssh | i@1PQD53Ux843x157553x0 | user | 2016-02-02 14:03:12,626 | SSH - git-receive-pack '/proj/repo.git' | - | - | - | - | - | - | os4we8 |atlassian-bitbucket.log
1
2016-02-02 14:03:13,050 INFO [NioProcessor-9] c.a.b.i.s.s.SelfClosingChannelSession Executing command: git-lfs-authenticate proj/repo.git uploadatlassian-bitbucket-access.log
1
2
3
4
192.168.1.10 | ssh | i@1PQD53Ux843x157554x1 | user | 2016-02-02 14:03:13,050 | SSH - git-lfs-authenticate proj/repo.git upload | - | - | - | - | - | - | 6pp10j |
192.168.1.10 | ssh | o@1PQD53Ux843x157554x1 | user | 2016-02-02 14:03:13,067 | SSH - git-lfs-authenticate proj/repo.git upload | - | 0 | 0 | 552 | ssh:user:id:1 | 17 | 6pp10j |
192.168.1.10,127.0.0.1 | https | i@1PQD53Ux843x157555x1 | - | 2016-02-02 14:03:13,575 | "POST /scm/proj/repo.git/info/lfs/objects/batch HTTP/1.0" | "" "git-lfs/1.1.0 (GitHub; darwin amd64; go 1.5.1)" | - | - | - | - | - | - |
192.168.1.10,127.0.0.1 | https | o@1PQD53Ux843x157555x1 | user | 2016-02-02 14:03:13,669 | "POST /scm/proj/repo.git/info/lfs/objects/batch HTTP/1.0" | "" "git-lfs/1.1.0 (GitHub; darwin amd64; go 1.5.1)" | 200 | 8907 | 12064 | - | 94 | 1d9nj3u |nginx
1
2016/02/02 14:00:46 [error] 16062#0: *1224429 client intended to send too large body: 21693741 bytes, client: 4.3.2.1, server: stash.example.com, request: "PUT /rest/git-lfs/storage/MAIN/repo/e79f7abda0ba90f2be4e613f78d1ecb7344970711106fe80d0028106f97f613c HTTP/1.1", host: "stash.example.com"In addition...
You can run a Git DEBUG. On the example we saw, the response by nginx was saying "too-large header".
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ GIT_TRACE=1 git push
14:44:49.161648 git.c:348 trace: built-in: git 'push'
...
trace git-lfs: tq: starting 3 transfer workers
trace git-lfs: tq: running as batched queue, batch size of 100
trace git-lfs: tq: sending batch of size 4
...
trace git-lfs: api: batch 4 files
trace git-lfs: HTTP: POST https://stash.example.com/scm/main/repo.git/info/lfs/objects/batch
trace git-lfs: HTTP: 200
trace git-lfs: HTTP: {"objects":[{"oid":"1bd91c03...62dbaa79","size":2746892,"actions":{}},{"oid":"58ef47926...70bb6ea89c","size":10639089,"actions":{}},{"oid":"2631c0...c6f6be538","size":2890471,"actions":{}},{"oid":"84c721...bc0a0f79","size":72016893,"actions":{"upload":{"href":"https://stash.example.com/rest/git-lfs/storage/MAIN/repo/84c72136
trace git-lfs: HTTP: e4d22c2...c0a0f79","header":{"Authorization":"JWT eyJ0e...NzA0NW
trace git-lfs: HTTP: IwOD...ifYKU"}}}}]}
trace git-lfs: HTTP:
Git LFS: (0 of 4 files, 3 skipped) 0 B / 84.20 MB, 15.52 MB skipped trace git-lfs: HTTP: PUT https://stash.example.com/rest/git-lfs/storage/MAIN/repo/84c72136e4...9c3bc0a0f79
trace git-lfs: HTTP: 413
trace git-lfs: HTTP: <html>
<head><title>413 Request Entity Too Large</title></head>
<body bgcolor="white">
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>nginx/1.9.3 (Ubuntu)</center>
</body>
</html>
...
Git LFS: (2 of 4 files, 3 skipped) 262.52 KB / 84.20 MB, 15.52 MB skipped
Client error: https://stash.example.com/rest/git-lfs/storage/MAIN/repo/84c7213...a0f79 from HTTP 413
error: failed to push some refs to 'ssh://git@stash.example.com:7999/main/repo.git'
$Resolution
This usually happens if you have a reverse proxy that does not allow a header that is too big. Please configure your HTTP proxy correctly.
The fix for nginx is to add/modify this server setting for the relevant vhost pointing to bitbucket server, modified for the maximum size of file you intend to store via git-lfs:
1client_max_body_size 256M;
Even if you perform Git LFS operations over SSH, they are actually done of HTTPS.
Was this helpful?