Upload to Azure Devops – This push was rejected because its size is greater than the 5120 MB limit for pushes in this repository.
During uploading fresh repository to GIT especially to Azure Devops – you can see:
This push was rejected because its size is greater than the 5120 MB limit for pushes in this repository.
The idea is to split across the commits. Here is a script that can do it automatically based on file size.
Invocation from directory of where you have repo: ..\push-one-dir.ps1 -RepoUrl “https://dev.azure.com/organisation/SVN/_git/gitrepo”
# Push repository in batches under 4500 MB
# Handles Polish/Unicode characters in filenames properly
# Run from the repository directory, e.g.: cd C:\svn\MNG then ..\push-one-dir.ps1
#..\push-one-dir.ps1 -RepoUrl “https://github.com/user/repo.git” -MaxBatchSizeMB 3000
#..\push-one-dir.ps1 -RepoUrl “https://github.com/user/repo.git” -InitializeRepo
param(
[Parameter(Mandatory=$true)]
[string]$RepoUrl,[Parameter(Mandatory=$false)]
[int]$MaxBatchSizeMB = 4500,[Parameter(Mandatory=$false)]
[switch]$InitializeRepo = $false
)# Ensure UTF-8 encoding for proper handling of Polish characters
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$PSDefaultParameterValues[‘Out-File:Encoding’] = ‘utf8’# Start logging
$timestamp = Get-Date -Format “yyyyMMdd_HHmmss”
$repoName = Split-Path -Leaf (Get-Location)
$logFile = “C:\svn\logs\push_${repoName}_${timestamp}.log”# Create logs directory if not exists
if (-not (Test-Path “C:\svn\logs”)) {
New-Item -ItemType Directory -Path “C:\svn\logs” | Out-Null
}Start-Transcript -Path $logFile
Write-Host “========================================” -ForegroundColor Cyan
Write-Host “Starting push for: $repoName” -ForegroundColor Cyan
Write-Host “Repository URL: $RepoUrl” -ForegroundColor Cyan
Write-Host “Max batch size: $MaxBatchSizeMB MB” -ForegroundColor Cyan
Write-Host “Log file: $logFile” -ForegroundColor Cyan
Write-Host “Started at: $(Get-Date)” -ForegroundColor Cyan
Write-Host “========================================” -ForegroundColor Cyan$maxBatchSizeBytes = $MaxBatchSizeMB * 1MB
# Configure Git for Unicode support
Write-Host “Configuring Git for Unicode support…” -ForegroundColor Yellow
git config core.quotePath false
git config i18n.commitEncoding utf-8
git config i18n.logOutputEncoding utf-8# Initialize repo if requested
if ($InitializeRepo) {
if (Test-Path .git) {
Write-Host “Removing existing .git folder…” -ForegroundColor Yellow
Remove-Item -Recurse -Force .git
}git init
git remote add origin $RepoUrl
}# First commit – just .gitignore if it exists
if (Test-Path .gitignore) {
Write-Host “Adding .gitignore…” -ForegroundColor Yellow
git add .gitignore
git commit -m “Initial commit – gitignore” 2>$null
git push -u origin master –force
Write-Host “.gitignore pushed successfully” -ForegroundColor Green
} else {
Write-Host “No .gitignore found, skipping initial commit” -ForegroundColor Yellow
# Create an empty initial commit to establish the branch
git commit –allow-empty -m “Initial commit”
git push -u origin master –force
}# Get all files using PowerShell’s Get-ChildItem (handles Unicode properly)
# This bypasses Git’s problematic path escaping
Write-Host “Scanning for files…” -ForegroundColor Yellow$currentPath = (Get-Location).Path
$allFileObjects = Get-ChildItem -Recurse -File -Force | Where-Object {
$_.FullName -notmatch ‘\\\.git\\’ -and
$_.FullName -notmatch ‘\\\.svn\\’ -and
$_.Name -ne ‘.gitignore’
}Write-Host “Found $($allFileObjects.Count) files to process” -ForegroundColor Cyan
# Check which files are already tracked by git
$trackedFiles = @{}
$gitLsFiles = git ls-files -z 2>$null
if ($gitLsFiles) {
$gitLsFiles -split “`0” | Where-Object { $_ -ne “” } | ForEach-Object {
$trackedFiles[$_] = $true
}
}Write-Host “Already tracked: $($trackedFiles.Count) files” -ForegroundColor Cyan
$currentBatchSize = 0
$currentBatchFiles = @()
$batchNumber = 1
$skippedFiles = @()
$failedFiles = @()
$totalFilesProcessed = 0foreach ($fileObj in $allFileObjects) {
# Get relative path
$relativePath = $fileObj.FullName.Substring($currentPath.Length + 1)
# Normalize path separators for git
$relativePath = $relativePath -replace ‘\\’, ‘/’# Skip if already tracked
if ($trackedFiles.ContainsKey($relativePath)) {
continue
}$fileSize = $fileObj.Length
# If single file exceeds limit, skip it and warn
if ($fileSize -gt $maxBatchSizeBytes) {
Write-Host “WARNING: Skipping ‘$relativePath’ – file too large ($([math]::Round($fileSize/1MB, 2)) MB)” -ForegroundColor Yellow
$skippedFiles += [PSCustomObject]@{
File = $relativePath
SizeMB = [math]::Round($fileSize/1MB, 2)
Reason = “Exceeds batch size limit”
}
continue
}# If adding this file would exceed limit, push current batch first
if (($currentBatchSize + $fileSize) -gt $maxBatchSizeBytes -and $currentBatchFiles.Count -gt 0) {
Write-Host “”
Write-Host “========================================” -ForegroundColor Green
Write-Host “Pushing batch $batchNumber ($([math]::Round($currentBatchSize/1MB, 2)) MB, $($currentBatchFiles.Count) files)…” -ForegroundColor Green
Write-Host “========================================” -ForegroundColor Green$batchFailed = @()
foreach ($batchFile in $currentBatchFiles) {
# Use the full path for adding to handle special characters
$fullPath = Join-Path $currentPath ($batchFile -replace ‘/’, ‘\’)# Use -LiteralPath style by passing to git add
$result = git add — “$batchFile” 2>&1
if ($LASTEXITCODE -ne 0) {
Write-Host ” WARNING: Failed to add ‘$batchFile’: $result” -ForegroundColor Red
$batchFailed += $batchFile
}
}if ($batchFailed.Count -gt 0) {
$failedFiles += $batchFailed
}# Check if there’s anything to commit
$status = git status –porcelain
if ($status) {
git commit -m “Batch $batchNumber – $($currentBatchFiles.Count) files”$pushResult = git push origin master 2>&1
if ($LASTEXITCODE -ne 0) {
Write-Host “ERROR: Push failed for batch $batchNumber” -ForegroundColor Red
Write-Host $pushResult -ForegroundColor Red# Try to recover – maybe the batch is too large for the server
Write-Host “Attempting to reset and continue…” -ForegroundColor Yellow
git reset HEAD~1
} else {
Write-Host “Batch $batchNumber completed at $(Get-Date)” -ForegroundColor Cyan
$totalFilesProcessed += ($currentBatchFiles.Count – $batchFailed.Count)
}
} else {
Write-Host “No changes to commit in batch $batchNumber” -ForegroundColor Yellow
}# Reset for next batch
$currentBatchSize = 0
$currentBatchFiles = @()
$batchNumber++# Small delay to prevent overwhelming the server
Start-Sleep -Milliseconds 500
}# Add file to current batch
$currentBatchFiles += $relativePath
$currentBatchSize += $fileSize
}# Push remaining files
if ($currentBatchFiles.Count -gt 0) {
Write-Host “”
Write-Host “========================================” -ForegroundColor Green
Write-Host “Pushing final batch $batchNumber ($([math]::Round($currentBatchSize/1MB, 2)) MB, $($currentBatchFiles.Count) files)…” -ForegroundColor Green
Write-Host “========================================” -ForegroundColor Green$batchFailed = @()
foreach ($batchFile in $currentBatchFiles) {
$result = git add — “$batchFile” 2>&1
if ($LASTEXITCODE -ne 0) {
Write-Host ” WARNING: Failed to add ‘$batchFile’: $result” -ForegroundColor Red
$batchFailed += $batchFile
}
}if ($batchFailed.Count -gt 0) {
$failedFiles += $batchFailed
}$status = git status –porcelain
if ($status) {
git commit -m “Batch $batchNumber – $($currentBatchFiles.Count) files (final)”$pushResult = git push origin master 2>&1
if ($LASTEXITCODE -ne 0) {
Write-Host “ERROR: Push failed for final batch” -ForegroundColor Red
Write-Host $pushResult -ForegroundColor Red
} else {
Write-Host “Batch $batchNumber completed at $(Get-Date)” -ForegroundColor Cyan
$totalFilesProcessed += ($currentBatchFiles.Count – $batchFailed.Count)
}
} else {
Write-Host “No changes to commit in final batch” -ForegroundColor Yellow
}
}# Verify what’s still untracked
Write-Host “”
Write-Host “Verifying repository status…” -ForegroundColor Yellow
$remainingUntracked = git status –porcelain | Where-Object { $_ -match ‘^\?\?’ }
$remainingCount = ($remainingUntracked | Measure-Object).Count# Summary
Write-Host “”
Write-Host “========================================” -ForegroundColor Cyan
Write-Host “SUMMARY” -ForegroundColor Cyan
Write-Host “========================================” -ForegroundColor Cyan
Write-Host “Total batches pushed: $batchNumber” -ForegroundColor Green
Write-Host “Total files processed: $totalFilesProcessed” -ForegroundColor Green
Write-Host “Remaining untracked: $remainingCount” -ForegroundColor $(if ($remainingCount -eq 0) { “Green” } else { “Yellow” })
Write-Host “Completed at: $(Get-Date)” -ForegroundColor Greenif ($skippedFiles.Count -gt 0) {
Write-Host “”
Write-Host “Skipped files (too large):” -ForegroundColor Yellow
$skippedFiles | Format-Table -AutoSize
}if ($failedFiles.Count -gt 0) {
Write-Host “”
Write-Host “Failed to add ($($failedFiles.Count) files):” -ForegroundColor Red
$failedFiles | ForEach-Object { Write-Host ” $_” -ForegroundColor Red }
}if ($remainingCount -gt 0) {
Write-Host “”
Write-Host “Still untracked files ($remainingCount):” -ForegroundColor Yellow
$remainingUntracked | Select-Object -First 20 | ForEach-Object {
Write-Host ” $_” -ForegroundColor Yellow
}
if ($remainingCount -gt 20) {
Write-Host ” … and $($remainingCount – 20) more” -ForegroundColor Yellow
}# Save remaining files to a separate log
$remainingLogFile = “C:\svn\logs\remaining_${repoName}_${timestamp}.txt”
$remainingUntracked | Out-File -FilePath $remainingLogFile -Encoding UTF8
Write-Host “”
Write-Host “Full list of remaining files saved to: $remainingLogFile” -ForegroundColor Cyan
}Write-Host “”
Write-Host “Log saved to: $logFile” -ForegroundColor CyanStop-Transcript
Before I suggest to invoke:
git config –global core.autocrlf false
git config –global core.longpaths true
git config —global http.postBuffer 524288000