For most of college, I’ve kept it simple: I’d create a directory in my home folder for each project, then eventually move older or inactive ones into ~/programming/. When I change devices or hit file size limits, I’ll compress and send things to my NAS.
This setup has worked pretty well so far. But now that I’m graduating and my projects keep stacking up, I’m starting to wonder if there’s a more efficient system out there.
Curious—how do you all organize and store your projects? Any tips or methodologies that have made your lives easier over time?
The only person I’ve talked to about this is my mentor who’s been programming since the 60s (started on the IBM 1620 and Bendix G15) and he just mostly keeps projects in directories in his home directory and uses his godly regular expressions skills to find things that way. Makes me wonder if I’m overthinking it…
I have a folder for my projects on root and within those projects I have my GitHub repos all contained within their own directory named the same as the project.
If I am learning something, I have a folder for the topic I am learning, and a logseq file with all of my notes. Then I have folders for my book references, one for video or audio references, and then a folder for my practice projects.
I just have all my active projects under $HOME/projects/ and projects from other people in $HOME/bin/
I have a dedicated directory with subdirectories for each project and that’s it
On my personal computer
~/Projects/<name>, you need to remember that real-life is not like college, you won’t be working on a new project every week. If you have more stuff than you can manage like this, you’ve bitten more than you can chew.On my work computer it’s a bit more complex, because I have to work with other people’s projects as well, so I have a
~/Workfolder and in it several folders by type of stuff, e.g.opsfor operational stuff such as scripts to deploy stuff or grant permissions,codefor servers (and client) code, etc. Also if I’m working on something specific that requires multiple repos I create a folder for that project with the repos inside.Whatever Cargo generates for me. If I use workspaces, then I put the subprojects to the root of the directory.
I think this is asking about where to keep projects, not how to organize them internally.
Ah, OK. I keep them in the Documents directory.
xd
Everything is in git, and I tend to use IntelliJ as an IDE. So my projects are all in ~/Ideaprojects/[PROJECTNAME]
I push every project I work on right away to my gitea instance. If I expect not to work on something for some time I just delete the local copy.
When I change devices or hit file size limits, I’ll compress and send things to my NAS.
Well, that sounds inconvenient.
Yeah, I really should start using Git for everything, but I’ve been working with a lot of large datasets recently (mostly EEG data). A big part of improving accuracy comes from cleaning the data, which is huge and takes a while to process. I could set up a local Git server to keep track of everything or just save the base data files and regenerate as needed, but on my current setup, that process can take anywhere from 2-6 hours depending on the task. So for now, I’ve just been managing everything locally to save time.
git LFS might be for you. If the data takes so long to reprocess I think it is fine to check it in (possibly using LFS).
When I change devices or hit file size limits, I’ll compress and send things to my NAS.
Whaaatt!?!!? That sounds like you don’t use git? You should use git. It is a requirement for basically any job and there is no reason to not use it on every project. Then you can keep your projects on a server somewhere, on your NAS if you want else something like github/gitlab/bitbucket etc. That way it does not really matter about your local projects, only what is on the remote and with decent backups of that you don’t need to constantly archive things from your local machine.
Yeah I think a local Git server would be good, will try our forgejo since people seem to like it— I’ve been using git for a lot of projects but not so much for large files and HW stuff since when using GitHub there are size limitations. Does seem like it would be freeing to be able to delete whatever I want from my workstation without worrying about losing stuff
Size limitations? In git?
What is the average size of your source code files?
Normally you’d never run out of space in git unless you’re committing large binary files.
I keep a root folder. On Windows it’s in c:\something on Linux it’s in /something
Under there I’ve got projects organized by language. This helps me organize nix shells and venvs.
Syncthing keeps the code bases and synced between multiple computers
I don’t separate work from home because they don’t live in the same realm.
Only home stuff in the syncthing.
I put stuff in places, then immediately forget where I put them.
“… and God said, let us make man in our own image.”
${HOME}/Projects/(Personal|Work)/<project name>
If either folder gets to busy I start to create projects Meta folders that normally corospond with a gitlab group.
I just blow out the folders with a good ol rm -rf ./ And git pull if I want to mess with it again.
I used to be put everything in ~/Programming at the top level. I later started grouping projects by type (JVM, Web etc.) in subfolders because it was getting hard to find things. This was synced with Nextcloud. However, I then at some point passed 2 million files (200GB) in said folder and decided to search for a better solution.
I ended up using a selfhosted Forgejo instance. It allows for easy code searching across all projects, tagging projects by topic and language, LFS, and has useful project management tools built-in.
I’ve seen a lot of talk about large file sizes. How can you realistically reach 200GB in text? That’s around 2*10^11 characters. Or do you guys store something else as well, like sqls of data or pictures/textures/models?
It consisted of tensors weights, datasets (which can reach several gigabytes), images, 3d models, and roughly 250+ programming projects with binaries, git without LFS and also a lot build files.
Nextcloud was able to sync it all, but syncing was getting so slow that I had to keep my new laptop running for almost an entire day to get all synced to it. It also wasn’t that great at excluding certain folders (like build cache folders or NPM package files), you would have to set up exclusions on each device separately. Another problem with Nextcloud sync was that it would sometimes duplicate projects after had been moved in a subfolder.
Similar question was recently asked here
I recommend when it comes to finding things to just use a fuzzy finder, such as fzf.
Building on this, I recommend zoxide instead of only fzfing or regexping.
For people who like to keep everything they ever create, like college students, you can use
z 18.04/1to get to a directory like~/hw/random-school/fresh-1/analysis-18.04/pset1.Lets you nest without fear.
(Also, about your question: I’ve personally used
~/git/<projname>/and~/git/<org>/<projname>at the same time – e.g.~/git/aur/fuzzel-git)
Hosting my own git server on my NAS made my life easier and better due to the new freedoms it offers. Backups are centralized, and I have all the space i need to keep any interesting code safe. I am using forgjeo now and highly recommend it. You can also use other front ends (or none and just ssh/filesystem) but forgjeo gives me artifacts (ie docker registry), code search, LFS, and more. With my own git server, my local filesystem only has what I am working on recently (or as my workstation space allows). My home folder has a folder for each version control system (git, pijul, svn). Inside of these i have 2 sub folders: <domain>/<repo name>
Some examples of different domains are: open, work, personal, dragonish. I do not separate what forge or remote service in the filesystem, this is a persona boundary.
I use git remote names and branches in each repo to handle what software forge and any upstream/maintainers i need to work with. As an example my work repos only get pushed to my work server (ie, only 1 git remote named origin set to my work’s server), but my open ones will go to forgjeo and github (i setup 2 git remotes, origin and github. origin in this domain goes to my forgjeo). If i have a need i go into some more git branching strategy which I do find has helped me over my life, but I think I am overthinking this post now! keep it all simple enough for what your needs are.
Def going to check this out— I think it will solve my biggest problem with GitHub which is just the number of large files I have to deal with— overthought comments are the best comments- thanks!
I also use a NAS as my primary git server, although I’ve never heard of Forgejo before, but it sounds really interesting and definitely something that I may look into. It sounds really handy to have on hand, especially with code search and LFS. So thanks for introducing that!
- Keep everything in an external git service. You can use third party services like Codeberg, GitLab, or GitHub, or host your own on your NAS.
- When you’re not working on a project and don’t think you’ll need to reference it for a while, just delete it from your laptop. The code always lives in git anyway.
In terms of local storage, I usually have everything in
~/projects/project-name, and I don’t have tiny file size limits because I don’t use FAT32 filesystems — that’s the default filesystem you usually get on USB sticks and external hard drives you buy. You have to format those drives to something like EXT4 (Linux) or NTFS (Windows) or you get stuck with FAT32 which has 2gb file sizes.You’ve forgotten about exFAT my dude. Nothing uses FAT32 anymore. All your usb drives will be exFAT.
That’s not been my experience. Lots of drives I’ve bought have been FAT32 out of the box.
What about exFAT? It overcomes FAT32’s limitation and is nearly readable on every OS and has way higher file size limits.
Edit: In case of external storage like USB/hard drive.
ExFAT is good for portable devices, but if you’re working with something internally, there’s no reason not to use EXT4 or NTFS.
