I've just test-migrated my CodeSharing project from SF to GH

Martin Holmes

30 Jul 2015 30 Jul '15

3:18 p.m.

Hi all, especially Raff, our migration manager. I just migrated my project from SF to GitHub in the following steps: === 1. PREPARATION === Do a freeze (not a problem for my tiny project). Make sure you have complete backups and checkouts. Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this: { "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" } (Obviously this needs a bit of research.) === 2. GETTING TICKETS ETC. OUT OF SF ========================== Do a full export of the SF project from here: https://sourceforge.net/p/[proj]/admin/export This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me: admin.json discussion.json tickets.json wiki.json It doesn't include the code, of course. === MOVING CODE TO GITHUB ===================================== Go here: https://import.github.com/new/ and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo: https://svn.code.sf.net/p/[proj] While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets). === MOVING TICKETS TO GITHUB ================================= I used this PERL script for this: https://github.com/cmungall/gosf2github Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it. There are several stages to this: 1. Set up an OAUTH token for the Git repo here: https://github.com/settings/tokens 2. Get a list of the project collaborators in JSON format from github like this: curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json 3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line: perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them. This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved. Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-) Cheers, Martin

Show replies by date

Raffaele Viglianti

30 Jul 30 Jul

3:19 p.m.

New subject: [tei-council] I've just test-migrated my CodeSharing project from SF to GH

Hi Martin, You beat me to it: I was working on exactly the same stuff with the same tools :) Raff On Jul 30, 2015 6:18 PM, "Martin Holmes" wrote:

...

Hi all, especially Raff, our migration manager.

I just migrated my project from SF to GitHub in the following steps:

=== 1. PREPARATION ===

Do a freeze (not a problem for my tiny project).

Make sure you have complete backups and checkouts.

Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this:

{ "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" }

(Obviously this needs a bit of research.)

=== 2. GETTING TICKETS ETC. OUT OF SF ==========================

Do a full export of the SF project from here:

https://sourceforge.net/p/[proj]/admin/export

This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me:

admin.json discussion.json tickets.json wiki.json

It doesn't include the code, of course.

=== MOVING CODE TO GITHUB =====================================

Go here:

https://import.github.com/new/

and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo:

https://svn.code.sf.net/p/[proj]

While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets).

=== MOVING TICKETS TO GITHUB =================================

I used this PERL script for this:

https://github.com/cmungall/gosf2github

Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it.

There are several stages to this:

1. Set up an OAUTH token for the Git repo here:

https://github.com/settings/tokens

2. Get a list of the project collaborators in JSON format from github like this:

curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json

3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line:

perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json

You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them.

This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved.

Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-)

Cheers, Martin

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Raffaele Viglianti

31 Jul 31 Jul

9:22 a.m.

New subject: [tei-council] I've just test-migrated my CodeSharing project from SF to GH

Hello, I'm customizing the script that Martin experimented with to get as much information as possible transferred over, such as adding labels for feature requests, bugs and "milestone" colours. I'd like your feedback on how we should keep track of the ticket creator and ticket assignee. As we discussed during our last council call, the imported issues are all created by the user importing them. We can set up a TEI-C user that will own all of the imported issues. The original assignee can be assigned to the imported issue only if they have a GitHub user (I could locate the GitHub user of 46 out of 88 contributing SF users). Otherwise the issue can be assigned to no one, everyone (any), or a default user. My question is: where should we keep track of the original owner and assignee? They could be stored - in the text body of the issue - as labels, e.g. sf_assignee:some_user and sf_owner:some_user Using labels would allow us to filter and search these imported issues effectively. But we'd end but with a large amount of labels (at most 88 for ownership plus 42 for unmatched assignees). And those labels should not be re-used after the importing process. Thoughts? Raff On Thu, Jul 30, 2015 at 6:19 PM, Raffaele Viglianti < raffaeleviglianti@gmail.com> wrote:

...

Hi Martin, You beat me to it: I was working on exactly the same stuff with the same tools :)

Raff On Jul 30, 2015 6:18 PM, "Martin Holmes" wrote:

...
Hi all, especially Raff, our migration manager.

I just migrated my project from SF to GitHub in the following steps:

=== 1. PREPARATION ===

Do a freeze (not a problem for my tiny project).

Make sure you have complete backups and checkouts.

Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this:

{ "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" }

(Obviously this needs a bit of research.)

=== 2. GETTING TICKETS ETC. OUT OF SF ==========================

Do a full export of the SF project from here:

https://sourceforge.net/p/[proj]/admin/export

This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me:

admin.json discussion.json tickets.json wiki.json

It doesn't include the code, of course.

=== MOVING CODE TO GITHUB =====================================

Go here:

https://import.github.com/new/

and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo:

https://svn.code.sf.net/p/[proj]

While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets).

=== MOVING TICKETS TO GITHUB =================================

I used this PERL script for this:

https://github.com/cmungall/gosf2github

Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it.

There are several stages to this:

1. Set up an OAUTH token for the Git repo here:

https://github.com/settings/tokens

2. Get a list of the project collaborators in JSON format from github like this:

curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json

3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line:

perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json

You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them.

This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved.

Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-)

Cheers, Martin

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Martin Holmes

10:24 a.m.

New subject: [tei-council] I've just test-migrated my CodeSharing project from SF to GH

Hi Raff, I like the idea of using labels as you suggest. I think it's probably worth sending a quick email to TEI-L to see if you can track down a few more SF users with GH accounts. After import, people who didn't respond will still be able to find all their tickets using the label with their SF id, and may be encouraged to get a GH account we can use. But I see no reason not to include the original owner and assignee in the text of the message too. I don't see a problem with a hundred or so additional labels; as soon as the import is done, we can set about tracking down those users and replacing labels with explicit assignments. If users don't want to create GH accounts, the label can be deleted and the info will still be there in the body of the ticket. For issues with no assignee in GH, I'd suggest using the same default creator (tei_orphan?). Cheers, Martin On 15-07-31 09:22 AM, Raffaele Viglianti wrote:

...

Hello,

I'm customizing the script that Martin experimented with to get as much information as possible transferred over, such as adding labels for feature requests, bugs and "milestone" colours.

I'd like your feedback on how we should keep track of the ticket creator and ticket assignee.

As we discussed during our last council call, the imported issues are all created by the user importing them. We can set up a TEI-C user that will own all of the imported issues.

The original assignee can be assigned to the imported issue only if they have a GitHub user (I could locate the GitHub user of 46 out of 88 contributing SF users). Otherwise the issue can be assigned to no one, everyone (any), or a default user.

My question is: where should we keep track of the original owner and assignee? They could be stored - in the text body of the issue - as labels, e.g. sf_assignee:some_user and sf_owner:some_user

Using labels would allow us to filter and search these imported issues effectively. But we'd end but with a large amount of labels (at most 88 for ownership plus 42 for unmatched assignees). And those labels should not be re-used after the importing process.

Thoughts? Raff

On Thu, Jul 30, 2015 at 6:19 PM, Raffaele Viglianti < raffaeleviglianti@gmail.com> wrote:

...
Hi Martin, You beat me to it: I was working on exactly the same stuff with the same tools :)

Raff On Jul 30, 2015 6:18 PM, "Martin Holmes" wrote:

...
Hi all, especially Raff, our migration manager.

I just migrated my project from SF to GitHub in the following steps:

=== 1. PREPARATION ===

Do a freeze (not a problem for my tiny project).

Make sure you have complete backups and checkouts.

Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this:

{ "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" }

(Obviously this needs a bit of research.)

=== 2. GETTING TICKETS ETC. OUT OF SF ==========================

Do a full export of the SF project from here:

https://sourceforge.net/p/[proj]/admin/export

This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me:

admin.json discussion.json tickets.json wiki.json

It doesn't include the code, of course.

=== MOVING CODE TO GITHUB =====================================

Go here:

https://import.github.com/new/

and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo:

https://svn.code.sf.net/p/[proj]

While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets).

=== MOVING TICKETS TO GITHUB =================================

I used this PERL script for this:

https://github.com/cmungall/gosf2github

Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it.

There are several stages to this:

1. Set up an OAUTH token for the Git repo here:

https://github.com/settings/tokens

2. Get a list of the project collaborators in JSON format from github like this:

curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json

3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line:

perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json

You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them.

This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved.

Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-)

Cheers, Martin

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Raffaele Viglianti

5 Aug 5 Aug

2:07 p.m.

New subject: [tei-council] I've just test-migrated my CodeSharing project from SF to GH

Hi all, So as Martin suggested I'm planning on sending an email to TEI-L asking for their GitHub usernames. I thought I'd share the body of the email here first since: a. I'm asking for potentially sensitive data; b. I'm somewhat making official our plans to at least move away from SourceForge; and c. I'm doing it in behalf of the council. Thoughts? See below. Dear all, The council is in the process of moving away from the SourceForge code repository as the hosting site is becoming alarmingly unstable and unreliable. We are considering a move to GitHub, but before we commit, we are making sure that the code history, bug reports, and feature requests can all be ported to a new platform without loss. If you have contributed to a bug report or feature request in SourceForge at any point, we need your help to keep an account that you own associated with your contribution. If you recognize your user name in the list below, please let me know your current GitHub user name (I'd recommend an off-list email). If you don't have a GitHub account, consider creating one. If you'd rather not creating a new account, no problem: we'll at least associate your SourceForge username with the migrated contribution. Let me stress that we are still in a phase of experimentation and there currently isn't any due date for migrating away from SourceForge. Thank you for your help! All the best, Raff Viglianti on behalf of the TEI council Here is the list of SourceForge user names who have contributed to a Bug Report of Feature Request. [LIST] On Fri, Jul 31, 2015 at 1:24 PM, Martin Holmes wrote:

...

Hi Raff,

I like the idea of using labels as you suggest. I think it's probably worth sending a quick email to TEI-L to see if you can track down a few more SF users with GH accounts. After import, people who didn't respond will still be able to find all their tickets using the label with their SF id, and may be encouraged to get a GH account we can use. But I see no reason not to include the original owner and assignee in the text of the message too. I don't see a problem with a hundred or so additional labels; as soon as the import is done, we can set about tracking down those users and replacing labels with explicit assignments. If users don't want to create GH accounts, the label can be deleted and the info will still be there in the body of the ticket.

For issues with no assignee in GH, I'd suggest using the same default creator (tei_orphan?).

Cheers, Martin

On 15-07-31 09:22 AM, Raffaele Viglianti wrote:

...
Hello,

I'm customizing the script that Martin experimented with to get as much information as possible transferred over, such as adding labels for feature requests, bugs and "milestone" colours.

I'd like your feedback on how we should keep track of the ticket creator and ticket assignee.

As we discussed during our last council call, the imported issues are all created by the user importing them. We can set up a TEI-C user that will own all of the imported issues.

The original assignee can be assigned to the imported issue only if they have a GitHub user (I could locate the GitHub user of 46 out of 88 contributing SF users). Otherwise the issue can be assigned to no one, everyone (any), or a default user.

My question is: where should we keep track of the original owner and assignee? They could be stored - in the text body of the issue - as labels, e.g. sf_assignee:some_user and sf_owner:some_user

Using labels would allow us to filter and search these imported issues effectively. But we'd end but with a large amount of labels (at most 88 for ownership plus 42 for unmatched assignees). And those labels should not be re-used after the importing process.

Thoughts? Raff

On Thu, Jul 30, 2015 at 6:19 PM, Raffaele Viglianti < raffaeleviglianti@gmail.com> wrote:

Hi Martin,

...
You beat me to it: I was working on exactly the same stuff with the same tools :)

Raff On Jul 30, 2015 6:18 PM, "Martin Holmes" wrote:

Hi all, especially Raff, our migration manager.

...
I just migrated my project from SF to GitHub in the following steps:

=== 1. PREPARATION ===

Do a freeze (not a problem for my tiny project).

Make sure you have complete backups and checkouts.

Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this:

{ "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" }

(Obviously this needs a bit of research.)

=== 2. GETTING TICKETS ETC. OUT OF SF ==========================

Do a full export of the SF project from here:

https://sourceforge.net/p/[proj]/admin/export

This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me:

admin.json discussion.json tickets.json wiki.json

It doesn't include the code, of course.

=== MOVING CODE TO GITHUB =====================================

Go here:

https://import.github.com/new/

and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo:

https://svn.code.sf.net/p/[proj]

While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets).

=== MOVING TICKETS TO GITHUB =================================

I used this PERL script for this:

https://github.com/cmungall/gosf2github

Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it.

There are several stages to this:

1. Set up an OAUTH token for the Git repo here:

https://github.com/settings/tokens

2. Get a list of the project collaborators in JSON format from github like this:

curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json

3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line:

perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json

You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them.

This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved.

Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-)

Cheers, Martin

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Hugh Cayless

2:15 p.m.

New subject: [tei-council] I've just test-migrated my CodeSharing project from SF to GH

creating -> create in ¶3. Otherwise, I think it’s fine. Technically it’s not a secret, since this is a public list :-)

...

On Aug 5, 2015, at 17:07 , Raffaele Viglianti wrote:

Hi all,

So as Martin suggested I'm planning on sending an email to TEI-L asking for their GitHub usernames. I thought I'd share the body of the email here first since: a. I'm asking for potentially sensitive data; b. I'm somewhat making official our plans to at least move away from SourceForge; and c. I'm doing it in behalf of the council. Thoughts? See below.

Dear all,

The council is in the process of moving away from the SourceForge code repository as the hosting site is becoming alarmingly unstable and unreliable. We are considering a move to GitHub, but before we commit, we are making sure that the code history, bug reports, and feature requests can all be ported to a new platform without loss.

If you have contributed to a bug report or feature request in SourceForge at any point, we need your help to keep an account that you own associated with your contribution.

If you recognize your user name in the list below, please let me know your current GitHub user name (I'd recommend an off-list email). If you don't have a GitHub account, consider creating one. If you'd rather not creating a new account, no problem: we'll at least associate your SourceForge username with the migrated contribution.

Let me stress that we are still in a phase of experimentation and there currently isn't any due date for migrating away from SourceForge.

Thank you for your help! All the best, Raff Viglianti on behalf of the TEI council

Here is the list of SourceForge user names who have contributed to a Bug Report of Feature Request.

[LIST]

On Fri, Jul 31, 2015 at 1:24 PM, Martin Holmes wrote:

...
Hi Raff,

I like the idea of using labels as you suggest. I think it's probably worth sending a quick email to TEI-L to see if you can track down a few more SF users with GH accounts. After import, people who didn't respond will still be able to find all their tickets using the label with their SF id, and may be encouraged to get a GH account we can use. But I see no reason not to include the original owner and assignee in the text of the message too. I don't see a problem with a hundred or so additional labels; as soon as the import is done, we can set about tracking down those users and replacing labels with explicit assignments. If users don't want to create GH accounts, the label can be deleted and the info will still be there in the body of the ticket.

For issues with no assignee in GH, I'd suggest using the same default creator (tei_orphan?).

Cheers, Martin

On 15-07-31 09:22 AM, Raffaele Viglianti wrote:

...
Hello,

I'm customizing the script that Martin experimented with to get as much information as possible transferred over, such as adding labels for feature requests, bugs and "milestone" colours.

I'd like your feedback on how we should keep track of the ticket creator and ticket assignee.

As we discussed during our last council call, the imported issues are all created by the user importing them. We can set up a TEI-C user that will own all of the imported issues.

The original assignee can be assigned to the imported issue only if they have a GitHub user (I could locate the GitHub user of 46 out of 88 contributing SF users). Otherwise the issue can be assigned to no one, everyone (any), or a default user.

My question is: where should we keep track of the original owner and assignee? They could be stored - in the text body of the issue - as labels, e.g. sf_assignee:some_user and sf_owner:some_user

Using labels would allow us to filter and search these imported issues effectively. But we'd end but with a large amount of labels (at most 88 for ownership plus 42 for unmatched assignees). And those labels should not be re-used after the importing process.

Thoughts? Raff

On Thu, Jul 30, 2015 at 6:19 PM, Raffaele Viglianti < raffaeleviglianti@gmail.com> wrote:

Hi Martin,

...
You beat me to it: I was working on exactly the same stuff with the same tools :)

Raff On Jul 30, 2015 6:18 PM, "Martin Holmes" wrote:

Hi all, especially Raff, our migration manager.

...
I just migrated my project from SF to GitHub in the following steps:

=== 1. PREPARATION ===

Do a freeze (not a problem for my tiny project).

Make sure you have complete backups and checkouts.

Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this:

{ "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" }

(Obviously this needs a bit of research.)

=== 2. GETTING TICKETS ETC. OUT OF SF ==========================

Do a full export of the SF project from here:

https://sourceforge.net/p/[proj]/admin/export

This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me:

admin.json discussion.json tickets.json wiki.json

It doesn't include the code, of course.

=== MOVING CODE TO GITHUB =====================================

Go here:

https://import.github.com/new/

and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo:

https://svn.code.sf.net/p/[proj]

While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets).

=== MOVING TICKETS TO GITHUB =================================

I used this PERL script for this:

https://github.com/cmungall/gosf2github

Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it.

There are several stages to this:

1. Set up an OAUTH token for the Git repo here:

https://github.com/settings/tokens

2. Get a list of the project collaborators in JSON format from github like this:

curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json

3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line:

perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json

You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them.

This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved.

Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-)

Cheers, Martin

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Martin Holmes

2:20 p.m.

New subject: [tei-council] I've just test-migrated my CodeSharing project from SF to GH

Just one more typo: a Bug Report of Feature Request => a Bug Report or Feature Request (at the end). Cheers, Martin On 15-08-05 02:07 PM, Raffaele Viglianti wrote:

...

Hi all,

So as Martin suggested I'm planning on sending an email to TEI-L asking for their GitHub usernames. I thought I'd share the body of the email here first since: a. I'm asking for potentially sensitive data; b. I'm somewhat making official our plans to at least move away from SourceForge; and c. I'm doing it in behalf of the council. Thoughts? See below.

Dear all,

The council is in the process of moving away from the SourceForge code repository as the hosting site is becoming alarmingly unstable and unreliable. We are considering a move to GitHub, but before we commit, we are making sure that the code history, bug reports, and feature requests can all be ported to a new platform without loss.

If you have contributed to a bug report or feature request in SourceForge at any point, we need your help to keep an account that you own associated with your contribution.

If you recognize your user name in the list below, please let me know your current GitHub user name (I'd recommend an off-list email). If you don't have a GitHub account, consider creating one. If you'd rather not creating a new account, no problem: we'll at least associate your SourceForge username with the migrated contribution.

Let me stress that we are still in a phase of experimentation and there currently isn't any due date for migrating away from SourceForge.

Thank you for your help! All the best, Raff Viglianti on behalf of the TEI council

Here is the list of SourceForge user names who have contributed to a Bug Report of Feature Request.

[LIST]

On Fri, Jul 31, 2015 at 1:24 PM, Martin Holmes wrote:

...
Hi Raff,

I like the idea of using labels as you suggest. I think it's probably worth sending a quick email to TEI-L to see if you can track down a few more SF users with GH accounts. After import, people who didn't respond will still be able to find all their tickets using the label with their SF id, and may be encouraged to get a GH account we can use. But I see no reason not to include the original owner and assignee in the text of the message too. I don't see a problem with a hundred or so additional labels; as soon as the import is done, we can set about tracking down those users and replacing labels with explicit assignments. If users don't want to create GH accounts, the label can be deleted and the info will still be there in the body of the ticket.

For issues with no assignee in GH, I'd suggest using the same default creator (tei_orphan?).

Cheers, Martin

On 15-07-31 09:22 AM, Raffaele Viglianti wrote:

...
Hello,

I'm customizing the script that Martin experimented with to get as much information as possible transferred over, such as adding labels for feature requests, bugs and "milestone" colours.

I'd like your feedback on how we should keep track of the ticket creator and ticket assignee.

As we discussed during our last council call, the imported issues are all created by the user importing them. We can set up a TEI-C user that will own all of the imported issues.

The original assignee can be assigned to the imported issue only if they have a GitHub user (I could locate the GitHub user of 46 out of 88 contributing SF users). Otherwise the issue can be assigned to no one, everyone (any), or a default user.

My question is: where should we keep track of the original owner and assignee? They could be stored - in the text body of the issue - as labels, e.g. sf_assignee:some_user and sf_owner:some_user

Using labels would allow us to filter and search these imported issues effectively. But we'd end but with a large amount of labels (at most 88 for ownership plus 42 for unmatched assignees). And those labels should not be re-used after the importing process.

Thoughts? Raff

On Thu, Jul 30, 2015 at 6:19 PM, Raffaele Viglianti < raffaeleviglianti@gmail.com> wrote:

Hi Martin,

...
You beat me to it: I was working on exactly the same stuff with the same tools :)

Raff On Jul 30, 2015 6:18 PM, "Martin Holmes" wrote:

Hi all, especially Raff, our migration manager.

...
I just migrated my project from SF to GitHub in the following steps:

=== 1. PREPARATION ===

Do a freeze (not a problem for my tiny project).

Make sure you have complete backups and checkouts.

Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this:

{ "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" }

(Obviously this needs a bit of research.)

=== 2. GETTING TICKETS ETC. OUT OF SF ==========================

Do a full export of the SF project from here:

https://sourceforge.net/p/[proj]/admin/export

This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me:

admin.json discussion.json tickets.json wiki.json

It doesn't include the code, of course.

=== MOVING CODE TO GITHUB =====================================

Go here:

https://import.github.com/new/

and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo:

https://svn.code.sf.net/p/[proj]

While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets).

=== MOVING TICKETS TO GITHUB =================================

I used this PERL script for this:

https://github.com/cmungall/gosf2github

Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it.

There are several stages to this:

1. Set up an OAUTH token for the Git repo here:

https://github.com/settings/tokens

2. Get a list of the project collaborators in JSON format from github like this:

curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json

3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line:

perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json

You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them.

This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved.

Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-)

Cheers, Martin

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Paul Schaffner

2:32 p.m.

New subject: [tei-council] I've just test-migrated my CodeSharing project from SF to GH

Looks good. (My SF = pfschaffner ; GH = PFSchaffner though I'm sure you knew that and I don't think they're case sensitive anyway) And lo, my email matches. And only my password manager knows what else. pfs PFSchaffner@umich.edu On Wed, Aug 5, 2015, at 17:20, Martin Holmes wrote:

...

Just one more typo:

a Bug Report of Feature Request => a Bug Report or Feature Request

(at the end).

Cheers, Martin

On 15-08-05 02:07 PM, Raffaele Viglianti wrote:

...
Hi all,

So as Martin suggested I'm planning on sending an email to TEI-L asking for their GitHub usernames. I thought I'd share the body of the email here first since: a. I'm asking for potentially sensitive data; b. I'm somewhat making official our plans to at least move away from SourceForge; and c. I'm doing it in behalf of the council. Thoughts? See below.

Dear all,

The council is in the process of moving away from the SourceForge code repository as the hosting site is becoming alarmingly unstable and unreliable. We are considering a move to GitHub, but before we commit, we are making sure that the code history, bug reports, and feature requests can all be ported to a new platform without loss.

If you have contributed to a bug report or feature request in SourceForge at any point, we need your help to keep an account that you own associated with your contribution.

If you recognize your user name in the list below, please let me know your current GitHub user name (I'd recommend an off-list email). If you don't have a GitHub account, consider creating one. If you'd rather not creating a new account, no problem: we'll at least associate your SourceForge username with the migrated contribution.

Let me stress that we are still in a phase of experimentation and there currently isn't any due date for migrating away from SourceForge.

Thank you for your help! All the best, Raff Viglianti on behalf of the TEI council

Here is the list of SourceForge user names who have contributed to a Bug Report of Feature Request.

[LIST]

On Fri, Jul 31, 2015 at 1:24 PM, Martin Holmes wrote:

...
Hi Raff,

I like the idea of using labels as you suggest. I think it's probably worth sending a quick email to TEI-L to see if you can track down a few more SF users with GH accounts. After import, people who didn't respond will still be able to find all their tickets using the label with their SF id, and may be encouraged to get a GH account we can use. But I see no reason not to include the original owner and assignee in the text of the message too. I don't see a problem with a hundred or so additional labels; as soon as the import is done, we can set about tracking down those users and replacing labels with explicit assignments. If users don't want to create GH accounts, the label can be deleted and the info will still be there in the body of the ticket.

For issues with no assignee in GH, I'd suggest using the same default creator (tei_orphan?).

Cheers, Martin

On 15-07-31 09:22 AM, Raffaele Viglianti wrote:

...
Hello,

I'm customizing the script that Martin experimented with to get as much information as possible transferred over, such as adding labels for feature requests, bugs and "milestone" colours.

I'd like your feedback on how we should keep track of the ticket creator and ticket assignee.

As we discussed during our last council call, the imported issues are all created by the user importing them. We can set up a TEI-C user that will own all of the imported issues.

The original assignee can be assigned to the imported issue only if they have a GitHub user (I could locate the GitHub user of 46 out of 88 contributing SF users). Otherwise the issue can be assigned to no one, everyone (any), or a default user.

My question is: where should we keep track of the original owner and assignee? They could be stored - in the text body of the issue - as labels, e.g. sf_assignee:some_user and sf_owner:some_user

Using labels would allow us to filter and search these imported issues effectively. But we'd end but with a large amount of labels (at most 88 for ownership plus 42 for unmatched assignees). And those labels should not be re-used after the importing process.

Thoughts? Raff

On Thu, Jul 30, 2015 at 6:19 PM, Raffaele Viglianti < raffaeleviglianti@gmail.com> wrote:

Hi Martin,

...
You beat me to it: I was working on exactly the same stuff with the same tools :)

Raff On Jul 30, 2015 6:18 PM, "Martin Holmes" wrote:

Hi all, especially Raff, our migration manager.

...
I just migrated my project from SF to GitHub in the following steps:

=== 1. PREPARATION ===

Do a freeze (not a problem for my tiny project).

Make sure you have complete backups and checkouts.

Prepare a JSON file called users_sf2gh.json which consists of the equivalent user names from SF and GitHub, like this:

{ "martindholmes" : "martindholmes", "stadlerpeter" : "peterstadler" }

(Obviously this needs a bit of research.)

=== 2. GETTING TICKETS ETC. OUT OF SF ==========================

Do a full export of the SF project from here:

https://sourceforge.net/p/[proj]/admin/export

This is not instant; you'll get an email when it's complete and you can download it as a zip. That produced these files for me:

admin.json discussion.json tickets.json wiki.json

It doesn't include the code, of course.

=== MOVING CODE TO GITHUB =====================================

Go here:

https://import.github.com/new/

and follow the instructions to import your code. This was quick for my tiny project, but would take a long time for the TEI codebase. The commit history is preserved correctly, as far as I can see. I provided the https URL for the SF repo:

https://svn.code.sf.net/p/[proj]

While that's going on, or after it's finished, you're offered the option to provide the list of SF/GitHub user equivalences you've already specified in the JSON file, through a web form (you'll use the JSON file itself later, when importing tickets).

=== MOVING TICKETS TO GITHUB =================================

I used this PERL script for this:

https://github.com/cmungall/gosf2github

Beware: there is a typo (occurring in two places) which omits a quotation mark in the example; I've added a patch and sent a pull request to fix it.

There are several stages to this:

1. Set up an OAUTH token for the Git repo here:

https://github.com/settings/tokens

2. Get a list of the project collaborators in JSON format from github like this:

curl -H "Authorization: token YOURTOKEN" https://api.github.com/repos/[entity]/[project]/collaborators > [project]-collab.json

3. Run the PERL script, which pulls down each ticket from SF and pushes it up to GitHub -- this is my command line:

perl gosf2github.pl -a martindholmes -u users_sf2gh.json -c codesharing-collab.json -r martindholmes/CodeSharing -s codesharing/tickets -t [MYTOKEN] tickets.json

You can run it with the -k flag first to do a dry-run and see the curl statements that would be addressed to github without sending them.

This successfully imported the two tickets I had created as a test, one open and one closed, with embedded XML code successfully preserved.

Hope this helps. It's nice to be on vacation. There's time to actually do stuff. :-)

Cheers, Martin

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived -- Paul Schaffner Digital Library Production Service PFSchaffner@umich.edu | http://www.umich.edu/~pfs/

3427

Age (days ago)

3433

Last active (days ago)

List overview

Download

7 comments

4 participants

participants (4)

Hugh Cayless
Martin Holmes
Paul Schaffner
Raffaele Viglianti