do you have any tips on getting 10DLC when you dont live in America? seems like using any virtual office is impossible they require an actual physical address /office
i recall very early in my career i did exactly this. i took what worked duplicated it—my reasoning being that it was far safer to reuse what has been battle tested and leave refactoring at a later stage
it wasn't received well and senior developer told me that 'good developers know exactly what patterns to use all the time before writing any piece of code and that he will clean up my mess'
long story short his refactoring caused what was otherwise a stable system into a complete mess and it reminded me of Nassim Taleb's book
It's definitely an "it depends" thing. It's easy to overabstract. On the other hand, I've also met junior developers who just didn't know how to use function parameters.
"use the right pattern" coming from a senior smells like a senior who can't freely design new patterns. Established wisdoms are a starting point, not the go-to solution.
I'm not the person you replied to, but I came from using VMware products for 12 years, to using Proxmox this last 1.5 years.
These are my impressions.
First of all it's a very competent product, mainly thanks to Ceph making it HCI. Without Ceph, I'm not sure what we would do.
It's as effective as you design it, make sure to separate storage and cluster traffic to ensure robustness, and speed. Make sure to use at least 10GbE switch for storage, for fast migrations.
And managing ceph is very important, basically boils down to 1) never let it run out of space, and 2) the more devices you have the easier it is to manage.
Automating against Proxmox definitely is the biggest pain point, and this needs the most work done.
I've spent countless hours, pre-AI, building our automation setup using both Terraform and Ansible. I sort of wish I had tried AI earlier because it does make things easier.
Some things like automating the creation of templates will forever be a complex procedure in Ansible. And I abandoned Terraform completely because the API was too unpredictable for Terraforms strict state, Ansible was a much better fit.
Their AuthZ takes some getting used to, the fact that if you select "Privilege Separation" it countes the user's permissions AND the token permissions, and the token permissions must always be lower than the users.
Templates existing on one node, but taking a unique VM ID across the cluster is also a bit confusing. It means in practice we're always deploying VMs on the same node, before migrating them somewhere else.
So you deploy a new VM from a template, shut it down, take a backup and then restore that backup to your target node. Is all this done with IaC? Ansible? Even the backup part?
Typically I don't touch templates once I do their initial setup, just shut them down and take a backup (can be done using ansible through the PVE CLI)
The backup restore and the VM startup is done through ansible > PVE CLI.
I also have a testing VM that has a "CLEAN" snapshot that I restore to multiple times a day, using ansible > PVE CLI. Once the VM snapshot is restored I turn it back on as well using the PVE CLI
TensorZero, LLMOps gateway, was archived yesterday and I forked it to continue development and keep it open source. I also applied for 6 months of codex credits which I will dedicate to the project.
I've built similar a couple of different ways. I'm currently focusing on running outputs through a local model to to enable tokenmaxing before sending onto a model I am paying with credits, not OAuth.
For now I will just open this up to see if there is any interest if so I would be spending whatever free time I get to fix issues and open it up to other contributors so we can keep going.
I think its important to have an LLM gateway tool like this to remain open source.
