Yes ,but No. Don’t
You are correct in that the driver will give the Victron system (that can be the Inverter and/or the MPPTs) the data and instruction of how much to charge and discharge the battery and when to stop.
But the Victron system is not responsable for the protection of the battery. That is the job of the BMS. And as such it can (and will) still charge/discharge in some instances even if the battery tells it not to. Some micro charges always flow. More details in this thread.
The short answer: Get the correct BMS and don’t bypass it’s protection. Burning down your house is not worth the saving for a smaller BMS and at 1/2 the price of 1 cell in your battery. Not worth it.