Adding a module to Kosmodular Grid - Wiki

[Sonosus]

Introduction

WARNING - BETA! PROCEED AT YOUR OWN RISK!

This guide should walk you through the process of getting one of your modules, PCBs, or stripboard designs listed on kosmodulargrid.com . It assumes no prior knowledge of any of the software tools or websites needed to upload your module. It tries to be as comprehensive as possible for popular operating systems. If there’s something you feel is missing, doesn’t make sense, or needs clarified, please feel free to edit this wiki and add it.

The Process

The basic process for adding your module is as follows:

1. Create a GitHub account and generate an API key.
2. Install Python, Git, and other dependencies on your computer.
3. Download and configure a copy of the module generator script.
4. Run the generator script.

Don’t worry if you don’t understand some of these steps - I’ll explain them in due course.

1. GitHub Setup

This step will give you the necessary credentials to connect with the kosmodulargrid database*. We’ll make a GitHub account and generate something called a ‘key’ or ‘token’ which is basically a unique password that lets a third party program log in to your account and carry out tasks on your behalf. In this case those tasks will be copying your module details into the online database and applying for permission from @jackb to make those changes go live on the site for the world to see. Let’s get started!

1. Go to github.com and either create an account or login. This is a fairly standard sign up process. Make sure you verify your email as some of the later steps won’t work if your account is unverified.

2. Once your account is set up, click your profile picture in the top right corner.
   

   

3. Click Settings.
   

   

4. At the bottom of the left panel, click Developer settings.
   

   

5. Click Personal access tokens.
   

   

6. Click Generate new token.
   

   

7. Set the Note to anything you like. This is a reference for you. I’m setting mine to kmg.
   

   

8. Set the token to never expire. GitHub will complain about this - ignore the warning.
   

   

9. Tick the boxes for repo , admin:repo_hook, and read:user.

10. Click the Generate token button.
    

    

11. Click the blue copy icon and paste the clipboard into a text file somewhere. Having access to this file gives anyone full write permissions to all your repositories and profile information. Keep this file safe and never share it with anyone. Make sure you copy the key now as you won’t be able to access it later on.
    

    

12. You’ve done the first stage! Take some time to sit back, relax, and grab a coffee, before we move on to step 2.

2. Python and Git.

This part is probably the most involved for those new to programming. We’ll need to download some software packages onto your computer. These packages will be:

• Git - this lets us copy the kosmodulargrid database and record our changes to it.
• Python - this is the programming language that we need to be able to run the script to modify the database.

I’ll split these instructions into sections for Windows and Linux. Unfortunately I don’t have a Mac to test the program on so I can’t write detailed instructions for you. Anyone with a Mac and knowledge of this process is welcome to contribute. Also, if you’re running a non Debian version of Linux, you’ll need to fend for yourself.

Installing on Windows

1. Go to Git - Downloading Package and download the latest version of Git for Windows.

2. Run the installer file.

3. Keep the default settings for most options and click Next. The options you should pay attention to are:

• Change the default editor to “Use Notepad as Git’s default editor”
• Make sure “Use Git from the command line and 3rd party software” is checked. This is very important.
• Change the default terminal emulator to “Use Windows’ default console window”
  a

4. Git should now be installed. To double check this, press the Windows key + R.

5. Type cmd and press return.

6. Type git --version. The terminal should respond with something like git version 2.34.1.

7. Now Git is installed we need to download and install Python.

8. Go to Python Releases for Windows | Python.org. At the time of writing, you’ll need to click on “Latest Python 3 release” then scroll down to the “Files” section, where you can then download either the 32-bit or 64-bit installer. (If you’re unsure, use this guide to help you.)

9.Run the installer file. Make sure you tick both of the boxes at the bottom of the window before clicking “Install Now”.

10. Python should now be set up. To test this, press Win+R then type cmd and press enter. Type py into the command window. It should return a screen like this:
    
    This verifies that Python is working and installed correctly.
    Type quit() and press return to get back to the command prompt.

11. Now, we need to use the cd command to change directory (switch folder) into our Documents folder, where our module files will be located.
    To do this, type in the command cd %homedrive%%homepath%\Documents, pressing return afterwards.

Installing on Linux(Debian based distros)

Hey, fellow Linuxians! You’ve got a much easier job than on Windows.

1. Open a terminal and type this command:
   sudo apt update && sudo apt install python3 python3-dev git python3-pip

2. Press y and enter when prompted.

3. Use the cd command to move into your Documents folder.

3. Configure the Module Generator

In this step we’ll download a copy of my module generator script that does a lot of complicated work behind the scenes so you don’t have to. You should open the command window you used earlier (you kept it open right?) and paste in each of these commands one by one, pressing enter after each one.

1. git clone https://github.com/sonosus/kmgenerator.git

2. cd kmgenerator

3. pip install -r requirements.txt (On some systems you may need to use pip3 instead of pip.)

4. Next, in your file manager, navigate to your Documents folder and open the kmgenerator folder.

5. Make a new file named key.txt and paste the token you generated earlier into it. Save this file.

After this, if you need to update the script to the latest version it should be sufficient to navigate to the kmgenerator folder and do:

1. git pull

2. pip install -r requirements.txt (On some systems you may need to use pip3 instead of pip.)

4. Run the Module Generator

Now we’re ready to run the generator script.

1. In the command prompt window (still open? good!), on Windows enter py fetchuuid.py and on Linux enter python3 fetchuuid.py.

2. If your name is listed in the terminal next to a long string of characters, enter it EXACTLY as it is shown on screen. Make sure you match upper and lower case characters and pay attention to spacing.
   

2b. If you haven’t got a name then enter your maker name here. This will appear on kosmodulargrid.com next to all your modules and will allow people to apply filters to find said modules. The script will ask you to enter your maker name again. Do so.
It will then ask for a short maker description for example LMNC forum user Analog Output.
After this is will ask for a maker website. If you don’t have a website then add your forum profile page or your GitHub profile. Make sure you add https:// to the start of the URL.

Press return to continue.

3. Enter the name of your module for example Kosmic Superspreader or 1222 VCO. This should be short. You’ll have an opportunity to elaborate on your module later on.

4. Enter a longer description of your module. This should be no longer than 2 lines and describe the function of your module, for example Dual voltage-controlled panner/crossfader with internal LFOs or Super simple noise generator with the Electric Druid Noise2 chip.
   (note: should we edit the modules that say “Kosmo format…” at the start seeing as it’s implied through the site name?)

5. Next enter the width of your module in centimetres, excluding the unit - enter 5 instead of 5cm.

6. Now we need a link to your module’s website.You could link to its GitHub page or your own website. MAKE SURE you add https:// to the start of the link else it might not work on all computers.

7. Add a link to the image for your module. If it’s hosted on GitHub, add ?raw=true to the end of the image address (which can be found through right click->copy image address on most browsers).

8. Now scroll back through the terminal and select your module’s function by entering the corresponding number from the list.

9. Choose the number that corresponds to your module’s type. (note: should we merge protoboard into stripboard?)

10. Press return 4 times until a prompt appears that asks if you want to create a second module. If you do, type Y and repeat steps 3–10 for the next module. If not, type N and the script will proceed to upload this data to GitHub and send a request to @jackb so he can add your changes to the main database. Unless you are using GitHub CLI or Git Credential Manager, you will be asked to authenticate by entering a username and “password”. Use the username you chose when you set up your GitHub account; for the “password” you actually need to enter the personal access token you generated. (It’s easiest to copy and paste it from the key.txt file.)

Code feedback and contributions are also much appreciated - I’m not a professional programmer so please don’t be too harsh

Source: GitHub - Sonosus/kmgenerator

*yeah I know it’s not an actual database but it’s the easiest way to describe it for non-coders.

[jackb]

Wow @sonusus, this script looks awesome! Nice work. I’ve been a little checked out of this forum lately, life just getting in the way. I’m looking forward to seeing some pull requests from this script!

[Sonosus]

Hopefully I didn’t spam you too much while I was testing it

[Sonosus]

@analogoutput

On my Linux Mint system, pip runs 2.7; if I want 3.x I have to use pip3.

Here on Ubuntu they both do the same thing.

@analogoutput

Aren’t you already there at this point?

In the terminal, yes, but in the graphical manager, no. I’m trying to keep it as user-friendly as possible and most users should already know how to edit a text file so I’d rather instruct them to do it graphically than teach more terminal commands.

@analogoutput

Could this have to do with the fact I already have a fork of kosmodulargrid?

I already have a fork and user.create_fork(repo) seems to ignore it. Try removing the newline at the end of the key file. I’ll see if I can have the script do this automatically.

If the script fails, make sure to delete the kmgenerator/kosmodulargrid folder to keep Git happy.

[Sonosus]

I think it’s standard practice to add a newline to every file for some historical reason.
I’m working on a fix.

[analogoutput]

Kosmo module that take input from brainwaves, runs them through a foo-bar algorithm on an Arduino Meta, and converts them to quantized 1.3v/oct notes via an 11.6 bit DAC.

Where can I get one??

[analogoutput]

I’ve deleted most of my replies above because I shouldn’t have made them here in the first place, I should have done them on Sonosus’s repo’s Issues tab.

What’s become clear is that, while this script can make adding a module easier — if everything works — it’s still a heavy lift. The user needs to set up a GitHub account, install Git, install Python, and get a GitHub PAT. Authentication to push the changes is still being worked out. It looks like it would require either manually entering the username and copying and pasting the PAT at the prompts every time, or installing and setting up GitHub CLI or Git Credential Manager, or setting up SSH authentication. All that is rather a lot to ask of someone who just wants to share their stripboard layout with the world. And if anything goes wrong, a naive user isn’t going to be able to fix it without a lot of back and forth with Sonosus or jackb or someone.

It seems to me this script, well intentioned as it is, is running on the wrong end. The onus ought to be on the kosmodulargrid end — the user could enter the same kind of information into a web form and KMG could one way or another get that information into its database.

That unfortunately doesn’t look like it could happen soon. As @jackb says, life has been getting in the way. Maybe someone else could help. I would, but I know next to nothing about this kind of development.

[Sonosus]

Very well said.

I think the main problem with KMG at the moment is that it’s all static. Anything with a web form that queries or updates a database needs a backend running on a paid server to host it, and a not insignificant amount of time and effort to set up.
I would offer help but again, Python is the only language I know to a reasonable level of competency (including StackExchange copypasta) so I don’t think I can be of much use.

Hopefully someone more technically inclined than myself can jump in to help.

[analogoutput]

I’ve submitted PRs to fix a couple of bugs in the script. With those fixes in place I succeeded in using it to push a PR with two new modules to KMG. Unfortunately at first the PR also contained some junk entries from my earlier testing. I had to manually remove them from modules.json. But that was basically just user error on my part, pushing those entries to GitHub and not removing them before proceeding.

(Authentication by manually entering username and PAT.)

[Sonosus]

PRs merged, thanks for cleaning those up.

I’ve had a few ideas on how the script could be adapted so no installation is needed. I’d rather not bake them into history in a GitHub issue so I’m floating them past more eyes here.

1. Use pythonanywhere.com and a bit of frontend development to make a web version of the script.

Pros:

• Stupid simple to use.
• No server admin needed.

Cons:

• With my front-end development skills, this would look horrific.
• I’d need to migrate away from cookie cutter because it’s designed for a user entering commands into a console. Something like copier could work.
• I also don’t know how much of the filesystem PythonAnywhere gives you access to - installing Git might not be possible.

2. Develop a Discourse chatbot like @discobot that asks the user to enter their module information and passes it to the script running on a VPS. I could probably host it on one of my DigitalOcean droplets for the time being.

Pros:

• Simple to use.
• Anyone on this forum can use it directly.
• Limited frontend development needed.

Cons:

• A paid server is needed with associated admin tasks and ‘downtime stress’.
  I could fix this by running it on my physical server in the garage but it’s already stretched for CPU cycles.
• I have no experience with Discourse development - it should work if there’s a Python library. I will experiment with chatbots on a local community forum I administrate.
• The bot would have to be given access to the forum database with potentially destructive consequences. I would conduct thorough testing on my on forums before releasing to the public.

3. A Discord chatbot on the LMNC discord server.

Pros:

• Free discord bot hosting is available for light usage like this
• I am fairly well versed in making Discord bots in Python and there’s a plethora of documentation everywhere you look.

Cons:

• Not everyone has a Discord account or wants to agree to the EULA
• Discord can be confusing to use
• Harder to find via search engine

With all these options, I’m leaning towards the idea of hard coding the PAT of a second GitHub account into the script. That account would then submit all pull requests rather than using the user’s credentials.

[discobot]

You’ve disabled me in your preferences. You need to allow new user onboarding tips to interact with me.

[Sonosus]

I’m leaning towards the Discourse chatbot idea. I won’t bore eveyone with the details, but I’ll be testing a plugin over here: https://forum.sonosus.me/ if anyone wants to follow my progress.

[Sonosus]

Quick update: this project grew many arms and legs quite quickly. I’ve almost finished converting the script to a Discord (not Discourse) bot ready for deployment on the LMNC Discord server. @Caustic I will DM you with details once it’s done, I’m just ironing out the bugs at the moment. A forum chat bot is a bit further down the pipeline; Discourse has a badly documented API with several abandoned attempts to turn it into a Python library, not ideal for a novice like me.

[Sonosus]

Kosmo's first words

Kosmo said his first words today. Now I need to polish off the API for kosmodulargrid itself and it’ll be ready for release.