Hello all fellow geeks!
Yeah I’m a proud geek!
After 2 or 3 weeks fiddling with making my own cloud at home, aka self hosted cloud,
and about to throw in the towel thinking, “I’m puny. I don’t have the skills for this! Give up buttercup!” …Knowing that my inner Yoda-nerd will never give up! I gave up on Google and other data harvester for a reason or five…
I ask this first:
Can I have anything on my LAN, on always on VPN, connect from outside via a domain name as xyzxyz.com??
My case:
I want to have my own “cloud” at home, for mainly share family photos with my family. Long story short, my mother and grandfather used a few summers, manually scanning 1000’s if not 10s of 1000’s pictures, all the way back to 1950s! As the family geek, I want to pay my respect to them 2, and offer to make a cloud for them, to share with the whole family.
I tested out TrueNAS Core, with NextCloud, and on my LAN, I love it! It have all the bells and whistles I want.
So I installed a physical server in the basement, with TrueNAS Core, and Nextcloud, copied over all pictures, and all is well.
Over the next 2-3 weeks, I follow between 20 and 50 different guides, on how to get this online.
Bought a domain, actually two, for two different providers, for trouble shooting reasons. Because… No matter what, none of my domains, connect to my home server…
What I have tried:
Both domains is now in Cludflare. Also because trouble shooting reasons.
The caveat if I cal call it… Is that my LAN is behind a pfSense box… ALL my LAN gadgets must go out of this house via VPN tunnels. Controlled by aliases. Gadgets not in correct alias, can’t exit this house.
Is THAT my problem??
Even though I have sat up DDNS for my Domains, Cloudflare get the correct IPS IP address for all my sub servers.
I have tried HAProxy clean. HAProxy with Virtial IP. Also in FW Rules, ALL ports open direct to my NextCloud server, still my xyzxyz.com addresses, with correct DDNS, with Challenges sat up, with Certs made through Acme certificates service. NOTHING helps!
I even installed a brand new bare metal Ubuntu Server, followed a guide to the dot, with install Nginx and Let’s encrypt bot.
Fail, fail fail. Can’t connect my server, Let’s encrypt bot can NEVER issue a cert, blaming this and that. I even open up for IPv6 in pfSense. No go!
Now that I wanted to say “GIVE UP, NERD!” In the shower, it hit me…
“Is it because my Server in the basement, is always on VPN???”
So I ask you gurus, that KNOW this stuff.
Must the server be off VPN, or do I have to make some special adjustment somewhere because my LAN Server is on a VPN?
ANY tips and tricks is highly appreciated!
Why is VPN always enabled?
Because I care too much about data harvesting and trying to be a bit privacy minded in this house…
Since wife and kids don’t care, I have to care for them, so everything on this LAN goes out via VPN’s on that pfSense.
Even 3 different VPN’s, depending on what that gadget do, even 2 different countries for streaming. (wife and kids like foreign stuff to watch)
You can prevent most of that data harvesting by using pfsense and packages properly. DNSSEC and pf-blocker ng are your friends. If the devices are using cookies for websites and services, which they most likely are, the harvesting is happening anyway, regardless of VPN use. That can only be mitigated from the client, if at all. Your issues with routing are likely due to how you’ve setup WAN, VPN and your NAT rules.
You’re right.
And yes I use pf-blocker too. It’s a God send.
My issue was port forwarding on always VPN. VPN tunnels need be configured for port forwarding.
So all works for me now.
Thanksgiving again 👍
How do you want to connect the internet to your server? You mentioned ddns, have you enabled port forwarding (you mention fw rules but dunno if you mean port forwarding)? Have you check if your isp actually lets you do port forwarding? What ip addresses you use, v4 or v6? Have you ever gotten a connection from the outside world to your server? Forget https or domain resolution for now, can you connect to it somehow?
Must the server be off VPN, or do I have to make some special adjustment somewhere because my LAN Server is on a VPN?
When you use a VPN service to reach the internet it makes the VPN a second WAN connection. It can cause problems if you don’t know what you are doing. You’ll probably need to use policy based routing (PBR) if you want to use the regular WAN (for incoming connections for the server) at the same time you want to use the VPN.
If your servers are on a vpn, the outside is not able to reach it, as there simply is no Network Address Translation going on outside the vpn. The computers might be able to reach the web, as pdfsense will open the ports required automatically and forward the requests, but it won’t do that for computers outside the vpn. In order for other computers to reach your vpn, the outside computer would also need to be a part of the vpn including the pdfsense as a gateway, only then can they communicate with each other.
Pdfsense was developed as a security tool to keep your internal network secure from outside computers.
If you want to reach your homelab via the web, you will probably need to use a normal router that supports wireguard or install a linux box where you can install wireguard or tailscale or netmaker etc and then open a UDP port for the wireguard port in your router.
This way you can make sure that you can setup the vpn the way you want, and not rely on what’s going on under your pdfsense hood. Depending on the pdfsense config there could be multiple reasons why it’s not doing what it does from lack of NAT to lack of iptable rules to other reasons as it simply not being configurated to act allow outside actors.
Thank you for a very good lesson! 👍
Thanks to you, and a couple of others, pointing out that being behind a VPN makes it impossible for NAT/port forwarding work, I must figure out how to config my pfSense, to accept ONE address exit without VPN…
The challenge is that this pfSense box was sat up 4-5 years ago, following a “Bullet proof” way of making it 100% sure, no LAN connected gadgets have ANY IP leaks what so ever.
And it works TOO well, now I can’t figure out how to reconfig it… I’m not skilled enough.
I have skills just enough as a 35 year IT veteran, to follow that guide, but tinker it to make one IP escape, is not in my powers, yet!Tried all tricks in my book, for let let’s say IP 10.0.0.201 exit this FW without a VPN, have been useless! Even in Rules LAN and WAN, giving that IP full access anywhere, and it can’t even ping google.com…
Apparently, I did (Or that VPN guide) a way too good job of securing this house hehe.
Normally I would be happy, but since I want to gift my mother with sharing family photos, same time as I don’t trust ANY online providers with privacy, “need” to self host this. And I am stuck as a n00b! :)
Thanks for trying to advice though
I’d recommend you ask in pfsense forum or irc channel. Those guys know the nitty gritty of it and also provide a diagram on draw.io so that they understand your configuration. Since pfsense seems to be the main gateway connecting all other computers, you need to fix it there first, before you can look at other possible locations to fix. Everything stands and falls with pfsense. Something that might seem impossible to an amateur might be solved trivially by an expert. Each and everyone have their domains so don’t feel bad or shy for asking. Nobody learned self hosting in a vacuum. We all share and learn. Since I have never used pdfsense I can’t help you with configuration, but I know the problem either lies with your router or pfsense.
If you need something simple that just works, use tailscale.com
I checked out them before with interest, but I can’t even sign up.
I don’t have any of them ID providers…
But thanks :)
The short answer to the question is yes. Three methods come to mind: