Veeam and ElasticSearch

[thomas_brun]

Hello guys

We currently have Veeam v10 installed (planning on upgrading to v11) on Server 2019 with Server 2019 Proxies on a VMware environment and backup with the virtual appliance transport mode.
Our mission is to backup everything with veeam as it makes everything easier to manage backups from a central backup solution.
Except for the ELK Stack (ElasticSearch, Logstash, Kibana), we achieve this goal. So my question is: Are there any plans to officially support backup of this product or does maybe anyone have experience on how to backup this solution with veeam?
The things about this topic which I found on google or here were not that satisfying, so I thaught I post my question here.

I guess there is currently no better solution than doing a backup from ELK itself somewhere to a server which is then backed up by veeam, but if there are any better solutions, I would be glad to hear about it.

Thanks in advance for your tips and advices and regards

Thomas

[Gostev]

Hello Thomas, can you please clarify what problems are you facing with backing up ELK using the current Veeam version? Thanks.

[thomas_brun]

Sure. The problem is that elasticsearch won't be backed up application aware / consistent since it is a clustered installation, but configured as active-active. To be honest, I never tested a backup / restore and what would happen because from what I've read is that this is not the correct way to do a file backup only (https://www.elastic.co/guide/en/elastic ... store.html) and it could result in data loss - and in worst case - corruption. For now, my elasticsearch co-worker is backing up the configuration and data to an smb share, but the server itself isnt backed up.

I've read that there are API's for elasticsearch cluster snapshots so I thaught that Veeam maybe has plans or intensions or already some solutions to backup ELK clusters, but I also know that elasticsearch is a niche product which is (at least currently) not widely spread and therefore nothing is planned for integration into veeam.

[mkretzer]

Hi,

it is definitely not a niche product! We have so many elasticsearchs we nearly lost count!

And Veeam backup is the reason we do not cluster them (more than one replica per index) most of the time.

So if there would be some way to cleanly backup ES clusters it would ne very nice!

Markus

[thomas_brun]

Alright, I withdraw the statement about the niche product

[salemone]

I am having a similar issue. I have been backing up my ElasticSearch VMs (Win 10) with Veeam and I did a test restore and I can't review logs. Thankfully this was a test. I will let you know if I figure anything out.

[HannesK]

Hello Andrew,
and welcome to the forums.

Just to clarify: you restored one VM of an Elasticsearch cluster and log replay (not sure what review means in this case) failed.

If you did that, it sounds expected for me (like with any distributed database where a "rollback" happened on one of the nodes).

Best regards,
Hannes

[salemone]

Hannes,

Each Elasticsearch instance is contained on 1 VM. I don't do anything with clustering or nodes. I have 2 VMs in 2 separate locations that don't communicate.

While doing some research, Elastic does not support a file level backup. They prefer to take a "snapshot" within Elastic/Kibana.

Andrew

[HannesK]

yes, there is a documentation on that https://www.elastic.co/guide/en/elastic ... store.html

[HannesK]

yep, there is a documentation on that https://www.elastic.co/guide/en/elastic ... store.html