KiCad library path munging

I recently realized that if you have a symbol or footprint library in KiCad that is below the project’s folder, the path gets stored as for instance ${KIPRJMOD}/subfolder/library.lib, but if it’s above the project’s folder it gets stored as an absolute path like /home/rsholmes/Documents/folder/library.lib. That’s rather annoying. For instance I often have a folder for a module containing two or more KiCad projects such as the PCB and the front panel, and I want them both to use the same library rather than two copies of (supposedly) the same library. But then if I move that folder, or if I put it up on GitHub, then that absolute path becomes garbage.

So I threw together the following Python script. You can run it from a KiCad project directory and give it fp-lib-table and sym-lib-table as arguments, and it’ll find any absolute paths in those files and convert them to paths relative to ${KIPRJMOD}, which it takes to be the current directory. (Or you can put ‘-p path’ in the command line and it’ll use the specified path instead of the current directory.) The original files are written to backups.

Then if you run KiCad it’ll look on those relative paths for the libraries instead of absolute paths.

If you specify -d num then it’ll only convert paths that are within that depth above the project directory. So you can convert any paths that are within the overall module’s folder structure while keeping any that point outside that structure as absolutes.

Yeah, error handling is a joke, but it should err on the side of harmlessness.

#!/usr/bin/python3

# Replace absolute paths in input KiCad files with paths relative to
# PWD or to specified (with -p option) path, by replacing path with
# ${KIPRJMOD}, its parent with ${KIPRJMOD}/.., etc.

# If -d num is specified, depth will be limited: Only paths that
# diverge num levels or fewer above PWD will be converted. (0 means
# only convert PWD, 1 means convert PWD or the directory above, etc.)

# E.g., for fp-lib-table in /home/rsholmes/Documents/DC_Mixer/PCB/ao_dc_mixer:
#
# (fp_lib_table
#  (lib (name ao_tht)(type KiCad)(uri ${KIPRJMOD}/ao_tht.pretty)(options "")(descr ""))
#  (lib (name ao_dc_mixer)(type KiCad)(uri "/home/rsholmes/Documents/DC_Mixer/Panel/ao_dc_mixer-panel/ao_dc_mixer.pretty")(options "")(descr ""))
#)
#
# becomes
#
# (fp_lib_table
#  (lib (name ao_tht)(type KiCad)(uri ${KIPRJMOD}/ao_tht.pretty)(options "")(descr ""))
#  (lib (name ao_dc_mixer)(type KiCad)(uri "${KIPRJMOD}/../../Panel/ao_dc_mixer-panel/ao_dc_mixer.pretty")(options "")(descr ""))
#)

from sys import argv
from os.path import dirname
from os import getcwd

def main():
    pwd = getcwd()
    depth = 9999
    
    findingpwd = False
    findingdepth = False
    for argi in argv[1:]:
        if findingpwd:
            findingpwd = False
            pwd = argi
        elif findingdepth:
            findingdepth = False
            depth = int (argi)
        elif argi == "-p":
            findingpwd = True
        elif argi == "-d":
            findingdepth = True
        else:
            try:
                print (depth)
                with open(argi, 'r') as ifile:
                    ifc = ifile.read()
                    ifcc = str(ifc)
                    tarp = pwd
                    rep = '${KIPRJMOD}'
                    d = 0
                    while tarp != '/' and d <= depth:
                        ifc = ifc.replace(tarp, rep)
                        tarp = dirname(tarp)
                        rep = rep + "/.."
                        d += 1
                if ifcc != ifc:
                    with open (argi+".bak", 'w') as ofile:
                        ofile.write(ifcc)
                    with open (argi, 'w') as ofile:
                        ofile.write(ifc)
            except:
                print ("Error with file", argi)

if __name__ == '__main__':
    main()
6 Likes

The library system is still a mystery to me. Also you and I must have different definitions of “munging” :rofl:

Cory,

I had to learn what it meant. It seems to multiple meanings in computer jargon related to “a series of potentially destructive or irrevocable changes to a piece of data or a file” and it is differentiated from munge. I learned mung is one of the first recursive acryonyms. It was originally “Mash Until No Good” but was revised to “Mung Until No Good.”

-Fumu / Esopus

1 Like

One person’s enhancement is another person’s munging.

2 Likes

have no f****ing idea what you are going on about here , but thank you Richard for taking the time for those that do understand and for all those other things that you explain to us , I have found a lot of it to be useful on my journey .

3 Likes

My libraries are stored on GIT and synced, not sure I have come across your problem but may not be paying attention. :wink:

I am very messy and have KiCad installed on a network drive, with the default libraries all over the place. I have a custom library with all my frequently used components (resistor, capacitor, pin header etc) in a new folder in KISYSMOD. I’ve never encountered this problem but it works for me, and while it might not be the correct way to do it, Kicad doesn’t complain.

Yeah I also have mine stowed away somewhere without much trouble. The real fun begins when you want to share kicad files (eg on Github) and need to include all dependencies. :wink::wink:

3 Likes

Yeah, usually it works locally, but as soon as you want to share a kicad project the trouble begins! I heard that kicad6 will improve on that though!

1 Like

Maybe somehow, but I did check and found that 5.99 also stores absolute paths for libraries not under the project file.

I think I did read that the symbols used in a schematic will be stored in the sch file, similar to how currently footprints are stored in the kicad-pcb file, so you don’t have to have any of the symbol libraries to work with a sch file. But that doesn’t help if the libraries are updated and you want to use the updated versions.

At least in my installation, KISYSMOD is /usr/share/kicad/modules, so it’s not user writeable (and shouldn’t be). On a multi user system presumably a sysadmin could make user writeable folders with suitable permissions inside KISYSMOD but that doesn’t seem like a good idea. I’m surprised there isn’t a KIUSRMOD variable pointing to e.g. ~/.config/kicad/modules or something. There is a KICAD_USER_TEMPLATE_DIR complementing KICAD_TEMPLATE_DIR but that seems to be the only user directory defined. I also just noticed there’s a KIGITHUB which is listed but undefined in my installation, no idea what that does. Ah, in 5.99 it’s defined as

https://github.com/KiCad.

Not relevant to this issue.

1 Like

There’s this

You can define path variables like KICAD_USER_LIBRARY. But if you share the project, whatever path variable names you use would have to be manually defined by whoever uses it, and they’re global — the same definitions are seen by all projects. A mess.

I mean, the user can just put your libraries in whatever location they want and then go into the manage libraries dialogs and change the paths to them, but it should be something easier.

1 Like