Bitbucketのレポジトリをbulk cloneする

Bitbucketの自分のレポジトリのListが取れていますから

前の記事で個々のcloneまではできています。Git Bashを使って、Repository Listから複数のclone_urlを拾って、bulk (一括)git cloneすればいいだけですね。言うのは簡単。

まずはcurlでlist repositoryのAPIを叩いてみましょう。下記でもちろんPATは一部伏せてあります。

curl --request GET   --url "https://api.bitbucket.org/2.0/repositories/doctornao"   --user "xxxxyyyy@gmail.com:ATATT3xFfGF0HS3MoJmfCY0736sRG7JW0LIersIapKS45B33jJE9JAojQiE1Hw3p4-WvndEoPhZi4_SCtDxDCFr0s3N8oA0Y_these_are_replacedQwrobh2nkVd6pLiwcsiMIcEyhgBnRkagvmVRE5mZRWNhCaN8JHH7VN-uClfb-Lw5-jQJCoA7o=42180EA9"

これで、結果がjsonで得られます。めちゃくちゃ長いですが、緑部分がPAT(修正済み)です。結果の例は、

{"values": [{"type": "repository", "full_name": "doctornao/bitbucket-create-repository", "links": {"self": {"href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository"}, "html": {"href": "https://bitbucket.org/doctornao/bitbucket-create-repository"}, "avatar": {"href": "https://bytebucket.org/ravatar/%7B45edcdec-b896-4d80-8772-76a590978a57%7D?ts=default"}, "pullrequests": {"href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/pullrequests"}, "commits": {"href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/commits"}, "forks": {"href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/forks"}, "watchers": {"href": 

これではわかりませんから、いつものjqで整形、

{
  "values": [
    {
      "type": "repository",
      "full_name": "doctornao/bitbucket-create-repository",
      "links": {
        "self": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository"
        },
        "html": {
          "href": "https://bitbucket.org/doctornao/bitbucket-create-repository"
        },
        "avatar": {
          "href": "https://bytebucket.org/ravatar/%7B45edcdec-b896-4d80-8772-76a590978a57%7D?ts=default"
        },
        "pullrequests": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/pullrequests"
        },
        "commits": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/commits"
        },
        "forks": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/forks"
        },
        "watchers": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/watchers"
        },
        "branches": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/refs/branches"
        },
        "tags": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/refs/tags"
        },
        "downloads": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/downloads"
        },
        "source": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/src"
        },
        "clone": [
          {
            "name": "https",
            "href": "https://DoctorNao@bitbucket.org/doctornao/bitbucket-create-repository.git"
          },
          {
            "name": "ssh",
            "href": "git@ssh.bitbucket.org:doctornao/bitbucket-create-repository.git"
          }
        ],
        "hooks": {
          "href": "https://api.bitbucket.org/2.0/repositories/doctornao/bitbucket-create-repository/hooks"
        }
      },
      "name": "bitbucket-create-repository",
      "slug": "bitbucket-create-repository"

最初のrepositoryの主要なところまで、で今抜き出したいフィールドは、clone_urlあたりなので、

  jq -r '.values[] | .links.clone[] | select(.name == "'$PROTOCOL'").href'

で受ければ良いです。従ってcurlの出力を上記のjqで受けて、

USERNAME="xxxxyyyy@gmail.com"
APP_PASSWORD="ATATT3xFfGF0HS3MoJmfCY0736sRG7JW0LIersIapKS45B33jJE9JAojQiE1Hw3p4-WvndEoPhZi4_changed_this_part_65pMc8cJtBcJfQwrobh2nkVd6pLiwcsiMIcEyhgBnRkagvmVRE5mZRWNhCaN8JHH7VN-uClfb-Lw5-jQJCoA7o=42180EA9"
PROTOCOL="https" # または "ssh"

# APIでリポジトリ一覧を取得してクローン
curl -u "$USERNAME:$APP_PASSWORD" \
  "https://api.bitbucket.org/2.0/repositories/doctornao" | \
  jq -r '.values[] | .links.clone[] | select(.name == "'$PROTOCOL'").href' | \
  while read -r repo_url; do
    echo "Cloning $repo_url..."
    git clone "$repo_url"
  done

実行すると、

$ ./butbucket-bulk-clone.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 32672  100 32672    0     0  36604      0 --:--:-- --:--:-- --:--:-- 36627
Cloning https://DoctorNao@bitbucket.org/doctornao/bitbucket-create-repository.gi...
Cloning into 'bitbucket-create-repository'...
remote: Enumerating objects: 22, done.
remote: Counting objects: 100% (22/22), done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 22 (delta 9), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (22/22), 10.05 KiB | 381.00 KiB/s, done.
Resolving deltas: 100% (9/9), done.
...ning https://DoctorNao@bitbucket.org/doctornao/alt-json-test.git
Cloning into 'alt-json-test'...
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 8 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (8/8), 6.60 KiB | 6.60 MiB/s, done.
Cloning https://DoctorNao@bitbucket.org/doctornao/jsoniterator-find-path-test.gi...
Cloning into 'jsoniterator-find-path-test'...
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 14 (delta 4), reused 14 (delta 4), pack-reused 0 (from 0)
Receiving objects: 100% (14/14), 9.65 KiB | 16.00 KiB/s, done.
Resolving deltas: 100% (4/4), done.
Cloning https://DoctorNao@bitbucket.org/doctornao/altjsoniterator-find-path-test...t
Cloning into 'altjsoniterator-find-path-test'...
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 14 (delta 4), reused 14 (delta 4), pack-reused 0 (from 0)
Receiving objects: 100% (14/14), 9.65 KiB | 9.65 MiB/s, done.
Resolving deltas: 100% (4/4), done.
Cloning https://DoctorNao@bitbucket.org/doctornao/jsoniterator-find-path-test-al...it
Cloning into 'jsoniterator-find-path-test-alt'...
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 14 (delta 4), reused 14 (delta 4), pack-reused 0 (from 0)
Receiving objects: 100% (14/14), 9.65 KiB | 9.65 MiB/s, done.
Resolving deltas: 100% (4/4), done.
...ning https://DoctorNao@bitbucket.org/doctornao/vcl-xaudio.git
Cloning into 'vcl-xaudio'...
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (5/5), 5.59 KiB | 5.59 MiB/s, done.
...ning https://DoctorNao@bitbucket.org/doctornao/vcl-xaudio2.git
Cloning into 'vcl-xaudio2'...
remote: Enumerating objects: 47, done.
remote: Counting objects: 100% (47/47), done.
remote: Compressing objects: 100% (46/46), done.
remote: Total 47 (delta 24), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (47/47), 13.66 KiB | 118.00 KiB/s, done.
Resolving deltas: 100% (24/24), done.
...ning https://DoctorNao@bitbucket.org/doctornao/speedbutton-test.git
Cloning into 'speedbutton-test'...
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 8 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (8/8), 30.22 KiB | 1.68 MiB/s, done.
...ning https://DoctorNao@bitbucket.org/doctornao/8led-stripe.git
Cloning into '8led-stripe'...
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 15 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (15/15), 8.78 KiB | 89.00 KiB/s, done.
Resolving deltas: 100% (2/2), done.
...ning https://DoctorNao@bitbucket.org/doctornao/vcl-hello-git-test.git
Cloning into 'vcl-hello-git-test'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 11 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (11/11), 6.01 KiB | 6.01 MiB/s, done.
Resolving deltas: 100% (2/2), done.

上記は”Git for Windows”のBashでの実行結果です。Linux上ではありません。結果は、

となります。スクリプト一発で取れています。これをGiteaへMigrateするには、GiteaのAPIを叩いて、Gitea側に同名のRepositoryをCreateして、ディレクトリに入りgit remote set-url origin ….して、最後にgit push -u origin main or masterすればいいですね。言うのは簡単ですが…..。bulk migrateは以降の記事で書くかもしれません。できなくはないですね。

コメント