↓Skip to main content
  1. 2024/
  2. Posts from August/

Restarting Failed Mirrored Repos in Gitlab

Stuff I think is interesting, cool, or otherwise worth sharing™️.

Today’s Inbox: 1,942 new unread messages
             ↪ Subject contains: ‘Repository mirroring paused’
                                     1242 messages found

(&!@#(&(!*@&#

Don’t you just hate it when background stuff breaks?

How to restart a bunch of failed mirrored repos in GitLab #

So… I mirror a lot of repos.. and, sometimes, when, for example, I have a networking issue, a bunch of mirrored repos can break….

This is kinda a pain in the ass to fix in the gui…

Fortunately, it’s not anywhere near as bad under the covers… GitLab’s Documentation1 is pretty easy to follow here:

How to get from A to Z #

Hop into the rails console: #

gitlab-rails console 
root@gitlab:/# gitlab-rails console
----------------------------------------------------------------------------
 Ruby:         ruby 3.2.5 (2024-07-26 revision 31d0f1a2e7) [x86_64-linux]
 GitLab:       17.5.1-ee (e4e13234b2d) EE
 GitLab Shell: 14.39.0
 PostgreSQL:   16.4

Look and see which projects have failed: #

gitlab-rails console 
Project.find_each do |p|
  if 
    p.group == mirrored_repos && 
    p.import_state && 
    p.import_state.retry_count > 14 && 
      puts "Mirroring failed #{p.import_state.retry_count} times for #{p.full_path}"
  end
end
gitlab-rails console 
Mirroring failed 15 times for mirrored_repos/repositoryName0
Mirroring failed 15 times for mirrored_repos/AppleApps/repositoryName1
Mirroring failed 15 times for mirrored_repos/question/repositoryName2
Mirroring failed 15 times for mirrored_repos/question/repositoryName3
Mirroring failed 15 times for mirrored_repos/coolify/repositoryName4
Mirroring failed 15 times for mirrored_repos/coolify/repositoryName5
Mirroring failed 15 times for mirrored_repos/truenas/repositoryName6
Mirroring failed 15 times for mirrored_repos/machinelearning/stability-ai/repositoryName7
Mirroring failed 15 times for mirrored_repos/the-bootcamp-project/configurations/repositoryName8
Mirroring failed 15 times for mirrored_repos/machinelearning/lmstudio/repositoryName9
Mirroring failed 15 times for mirrored_repos/machinelearning/janhq/repositoryName10
…You get the point

Reset the failure counts: #

gitlab-rails console 
Project.find_each do |p|
  if p.import_state && p.import_state.retry_count >= 14
    puts "Resetting mirroring operation for #{p.full_path}"
    p.import_state.reset_retry_count
    p.import_state.set_next_execution_to_now(prioritized: true)
    p.import_state.save!
  end
end

  1. https://docs.gitlab.com/ee/user/project/repository/mirror/pull.html#fix-hard-failures-when-mirroring ↩︎