Feature req: embed postgres as default database, not sqlexpr

Availability for the Always-On Enterprise

Feature req: embed postgres as default database, not sqlexpr

Veeam Logoby aceit » Mon Aug 14, 2017 5:09 pm

Hello,
it would be great if Veeam in the future would consider the possibility to use in the default install the postgres sql database and not the limited sql express, transparently.

Vmware already, for its most critical components like virtual center server, transitioned to PostgreSQL as the default option, with many benefits in the present and in the future, like:

- predictability of the db software stack used by customers (less support and dev costs)
- optimizations and speed (due to predictability, dev use the same system as customer)
- no limits (sqlexpress limits become a no-problem), almost infinite scalability
- no direct or separate licence costs
- high flexibility (replications, clustering, ha... etc... see what vmware can do with the embedded postgresql in the virtualcenter vm appliance...)
- high flexibility in programming and querying (ie customer can run custom sql query via jdbc and various automation tasks)

Personally, I can see only benefits (to veeam and to customers).

Just my 2c, glad if you can discuss and consider it
aceit
Influencer
 
Posts: 17
Liked: 5 times
Joined: Tue Jun 20, 2017 3:17 pm

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby PTide » Wed Aug 16, 2017 11:28 am

Hi,

predictability of the db software stack used by customers
I'm not sure what do you put into the term? Please elaborate.

optimizations and speed (due to predictability, dev use the same system as customer)
What developers are you referring to?

no limits (sqlexpress limits become a no-problem), almost infinite scalability
This one might be true. Have you hit the limit already? Please describe your configuration.

no direct or separate licence costs
True, there is no price tag, however there is such thing as "maintenance costs".

high flexibility (replications, clustering, ha... etc... see what vmware can do with the embedded postgresql in the virtualcenter vm appliance...)
Again, maintenance costs. Also, if I'm not mistaken, all these things can be done with MSSQL too.

high flexibility in programming and querying (ie customer can run custom sql query via jdbc and various automation tasks)
Customers are not supposed to directly interact with Veeam's back-end database. All manipulations with DB should be done via support team only.

Thank you!
PTide
Veeam Software
 
Posts: 3079
Liked: 252 times
Joined: Tue May 19, 2015 1:46 pm

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby aceit » Wed Aug 16, 2017 6:35 pm 1 person likes this post

Hello,
thanks for the feedback.
I want of course to make clear that this is just general input and idea, not dictated by urgency or my particular deployment or particular system problem or critics to MSSQL (that is a very good database per se) and I know that veeam is chiefly on windows and on not a *nix appliance (where postgres excels...).

Predictability means that, once standardized the db layer, and because it is a component under the hood and opaque to the customer, all customers will use exactly the same version given a particular Veeam version. Less configuration differences in the field, usually less costs and more benefits (tuning etc.). Because there is no artificial limit in the db per se, having an option to choose another "bigger" DBMS is not necessary (same engine for all, big or small). So less complexity. Same procedures and software level for all.
I know that veeam is not I/O hungry on the db layer per se, so many customers are perfectly fine with the current embedded MSSQL. Still, the aforementioned reasons are for example what guided vmware in embedding vpostgres directly in vcenter (yes, I know that there is a difference in domain and use case of the two solutions, but vmware is already collecting the benefits...: same integrated db engine for all the customer, less differences in the field, less support costs due to db specific issues, supported backups and procedures, optimizations done for all etc.).

again, just my input...
aceit
Influencer
 
Posts: 17
Liked: 5 times
Joined: Tue Jun 20, 2017 3:17 pm

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby alexander sh » Fri Aug 18, 2017 12:36 pm 5 people like this post

Hey aceit!
Here's my 2c as a VBR developer. I'm not gonna touch on any internal/organizational questions, just technical side of things.

While I can agree with some of your arguments (mainly regarding SQL server express limitations and general lack of edition constraints in Postgres), and while I think Postgres is a great DBMS with some fantastic programmability features, here's a few thoughts on what kind of challenges we'd have adding postgres support:

Adding Postgres as a second supported DBMS alongside SQL Server is an option that can be benificial for some customers, but it would not achieve anything on your list (" less differences in the field, less support costs due to db specific issues, supported backups and procedures, optimizations done for all"), rather the opposite. We'd end up supporting 2 different DBMS with slightly but annoyingly different SQL syntax (so essentially having to write and test all DB-side code twise), their respective performance quirks, possible corner cases in C# client support etc.

So the only way to achieve anything in terms of configuration uniformity and development/testing speed would be to swap SQL Server for Postgres and drop support for SQL Server altogether.
Dropping SQL server support - leaving aside the fact that SQL Server is a fantastic DB and many of our developers have a lot of experience with it etc. etc. - is hardly an option. Besides the amount of DB-specific code we'd need to rewrite and retest, migrating critical software for 250k+ customers between DBMSs sounds like a recipe for a lot of problems and confusion for customers: consider that many of our bigger customers host Veeam DBs on a dedicated fully lisenced SQL Servers alongside other production/infra DBs, have a dedicated SQL Server DBA and so on. At certain scale, hosting a DB on a VBR server may become a bottleneck, so these customers would now need to set up and tune their own Postgres servers.

I obviously can't speak for our Product Management team, but from development point of view adding support for Postgres not a small undertaking that would require very serious reasons to commit to.

On a bright side, SQL Server Standard edition is not really expensive, and it's all you really need for VBR even at scale.
Besides, Microsoft lately has been doing a lot of good stuff in terms of licensing, enabling many Enterprise-only features in all editions starting with SQL 2016 SP1 (https://blogs.msdn.microsoft.com/sqlrel ... -released/). We don't use many of those featues yet, but may start in future, now that they have become much more available to everybody.
alexander sh
Veeam Software
 
Posts: 23
Liked: 16 times
Joined: Thu Dec 13, 2012 8:52 am
Full Name: Alexander Shelemin

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby chris400 » Mon Aug 21, 2017 6:03 am

I dfntly like non-MS Software, but they have another big advantage: Patching is integrated into the OS and done automatically in default configuration.

I don't have much knowledge regarding the backend of veeam: something is done in xml, some in sql. Please explain what is done in xml and what in sql and why it is done in one or the other 8-)
chris400
Influencer
 
Posts: 10
Liked: 1 time
Joined: Sat Feb 04, 2017 10:31 am
Full Name: Christoph

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby alexander sh » Mon Aug 21, 2017 11:12 am

Chris400, to be honest, not sure what you mean by "done in xml"? Xml is just a data description language/format. We use xml as a format to store backup metadate (.vbm) files on disk, for example.
If you have any specific questions, I'll be happy to answer :)
alexander sh
Veeam Software
 
Posts: 23
Liked: 16 times
Joined: Thu Dec 13, 2012 8:52 am
Full Name: Alexander Shelemin

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby voyager529 » Tue Aug 22, 2017 6:01 pm

I'll throw my hat into the ring on this one, but for a very different reason...

I would love nothing more than for Veeam B&R to be a cross-platform title. At present, there are only two applications in my environment that run on Windows, and the other one is schedule for depreciation this year. I submit that allowing Postgres or MariaDB as a database backend would make it "a bit more possible" for there to Veeam to become independent of Windows Server. Veeam having a Docker container or self-contained OVA appliance would be the icing on the cake for an already excellent product.

Thank you for the consideration on this front.

Joey
voyager529
Influencer
 
Posts: 12
Liked: 2 times
Joined: Thu May 14, 2015 8:41 pm
Full Name: Joey Famiglietti

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby jmmarton » Tue Aug 22, 2017 6:04 pm

In theory when SQL for Linux is released you'll be able to keep using SQL Server yet still remain somewhat independent of Windows Server.

Joe
jmmarton
Veeam Software
 
Posts: 846
Liked: 89 times
Joined: Tue Nov 17, 2015 2:38 am
Location: Chicago, IL
Full Name: Joe Marton

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby aceit » Wed Aug 23, 2017 5:16 pm

thanks @alexander for the explanation in veeam actual usage of the db layer and veeam software engineering practices.
I understand that considering a db technology swapping would be at the present state not justifiable, given economics and the high coupling of veeam software layer to MSSQL stack.
Mine was just an input to poke into the eventual roadmap, given the current trend I see in the industry (ie standardization and less reliance on windows for infrastructure vm based components).

As some suggests here, having a self contained OVA appliance like VMware did for Vcenter indeed would be nice... still, as was explained by Veeam, it would complicate their developments (so > costs for users) with maybe little benefits... for example veeam still requires a separate physical server and thus a OS for for many "non-hypervisor-intrusive" (ie. san direct etc.) backup designs and tape libraries (so the whole set of complications like hw certifications, multipath drivers support etc. already solved by the OS vendor).... but yea for some designs having a *nix based deployable self contained OVA or similar of VBR on top of the hypervisor would be nice...

bye
aceit
Influencer
 
Posts: 17
Liked: 5 times
Joined: Tue Jun 20, 2017 3:17 pm

Re: Feature req: embed postgres as default database, not sql

Veeam Logoby chris400 » Sun Aug 27, 2017 10:03 pm

alexander sh wrote:Chris400, to be honest, not sure what you mean by "done in xml"? Xml is just a data description language/format. We use xml as a format to store backup metadate (.vbm) files on disk, for example.
If you have any specific questions, I'll be happy to answer :)

Sorry, I'm not a native speaker 8)

The backup itself is written to your propretary vib-file format.
What data is written to SQL? Veeam Agent for Microsoft Windows requires it even if only used locally, while Veeam Agent for Linux doesn't.
chris400
Influencer
 
Posts: 10
Liked: 1 time
Joined: Sat Feb 04, 2017 10:31 am
Full Name: Christoph


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], tdewin, the_mentor, tommyo and 47 guests