-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 11 posts ] 
Author Message
 Post subject: Performance of Hibernate Validator
PostPosted: Wed May 31, 2017 4:22 pm 
Newbie

Joined: Wed May 31, 2017 4:16 pm
Posts: 2
We defined 2 endpoints one validating the request body and the other one without the valid annotation.Having 20.000 articles(all valid) in the shop and calling both rest endpoints we get in these times:
/test/spring -> 62.145 ms
/test/own -> 11.35 ms
(these are average times of 200 runs)

Why is the rest endpoint using the Hibernate validator so much slower? Can we do anything better so that we can use the hibernate validator for a lot of data?


@RequestMapping(path = "/spring", method = RequestMethod.POST)
public ResponseEntity<Object> postSpringValidation(@RequestBody **@Valid** Shop shop) {
return new ResponseEntity<>(HttpStatus.CREATED);
}


@RequestMapping(path = "/own", method = RequestMethod.POST)
public ResponseEntity<Object> postOwnValidation(@RequestBody Shop shop) {
if (!shop.isValidData()) {
throw new InvalidParameterException("invalid data");
}

return new ResponseEntity<>(HttpStatus.CREATED);
}

public class Shop {
@NotNull
private String id;

@Valid
@NotNull
private List<Article> articles;

public boolean isValidData() {
if (id == null)
return false;

for (Article article : articles) {
if (!article.isValidData()) {
return false;
}
}

return true;
}
}

public class Article {
@NotNull
private String id;

public boolean isValidData() {
if (id == null)
return false;

return true;
}
}


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Mon Jun 05, 2017 8:25 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Hi,
200 calls is not much at all to be conclusive, also you should not look at the average. Was the JVM warmed up ?

Could you share which version it is? The most recent versions have been optimised significantly, it might be worth trying to upgrade Hibernate Validator.

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Tue Jun 06, 2017 2:59 am 
Newbie

Joined: Wed May 31, 2017 4:16 pm
Posts: 2
Hi
thanks for the reply.

warm up was done. We tested also with 1000 runs. Why should we not look at the average?
I tested with 5.4.1.Final and 6.0.0.Beta2

Is the Hibernate Validator suitable for high volume of data?

Regards


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Tue Jun 06, 2017 5:53 am 
Hibernate Team
Hibernate Team

Joined: Sat Jan 24, 2009 12:46 pm
Posts: 388
Hi, that difference looks suspicious. Any chance you attach a profiler and check where most time is spent?

I'm wondering how the validator is set up, I hope that it does not create a new Validator Factory instance for each call (which is costly as it detects all the constraint metadata). Validating such small tree should definitely not result in such a significant increase of request processing time, but it's hard to tell what's going on without having more details. Btw. HV 6 should be faster than 5.4, so if you see no difference between the two, it's also an indicator that the time is lost somewhere else.

Would love to have some more details on this one.

--Gunnar

_________________
Visit my blog at http://musingsofaprogrammingaddict.blogspot.com/


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Tue Jun 06, 2017 6:25 am 
Newbie

Joined: Tue Jun 06, 2017 6:16 am
Posts: 4
Hello,

Below you can find the difference between the own validation and the validation used by Spring Boot (@Valid using the Hiberante Validator):
Image

Below you can find the call stack from the profiler (2 invocations with 20000 articles per shop and a warmed-up JVM):
Image


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Tue Jun 06, 2017 6:30 am 
Newbie

Joined: Tue Jun 06, 2017 6:16 am
Posts: 4
Gunnar wrote:
I'm wondering how the validator is set up, I hope that it does not create a new Validator Factory instance for each call (which is costly as it detects all the constraint metadata).

We also tested it with an own Validator, the results were comparable to the usage of the @Valid annotation provided by Spring Boot:
Code:
   public Controller() {
      final ValidatorFactory validatorFactory = Validation.buildDefaultValidatorFactory();
      validator = validatorFactory.getValidator();
   }

   @RequestMapping(path = "/validator", method = RequestMethod.POST)
   public ResponseEntity<Object> postValidator(@RequestBody Shop shop) {
      final Set<ConstraintViolation<Shop>> violations = this.validator.validate(shop);

      if (!violations.isEmpty()) {
         throw new InvalidParameterException("invalid data");
      }

      return new ResponseEntity<>(HttpStatus.CREATED);
   }


We also tested the fail-fast mode, but as the input data is always valid there were no differences.


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Tue Jun 06, 2017 9:20 am 
Hibernate Team
Hibernate Team

Joined: Fri Oct 05, 2007 4:47 pm
Posts: 2536
Location: Third rock from the Sun
Could you check if that ValidatorFactory being built in the Controller() constructor is being reused?

You should make sure you have only one ValidatorFactory to handle all your incoming requests. From the look of your Controller I suspect you might have a new one created for each request, but I'm not sure as that will depend on your framework and configuration. I never used Spring Boot myself.

BTW I can't read the images from the profiler, they are way too small. I believe you only linked to the thumbnails?

Thanks

_________________
Sanne
http://in.relation.to/


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Tue Jun 06, 2017 11:02 am 
Newbie

Joined: Tue Jun 06, 2017 6:16 am
Posts: 4
sanne.grinovero wrote:
Could you check if that ValidatorFactory being built in the Controller() constructor is being reused?


I changed the code to the following:
Code:
public class ValidatorFactorySingleton {
   private static ValidatorFactory instance = null;

   private ValidatorFactorySingleton() {
   }

   public static ValidatorFactory getInstance() {
      if (instance == null) {
         instance = Validation.buildDefaultValidatorFactory();
      }

      return instance;
   }
}


Code:
   public Controller() {
      validator = ValidatorFactorySingleton.getInstance().getValidator();
   }



The results stay the same.


sanne.grinovero wrote:
BTW I can't read the images from the profiler, they are way too small. I believe you only linked to the thumbnails?

Here are the raw links:
http://abload.de/img/diff63uvb.png
http://abload.de/img/hibernate3zu47.png


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Wed Jun 07, 2017 6:12 am 
Hibernate Team
Hibernate Team

Joined: Thu Jan 05, 2017 6:04 am
Posts: 13
Hi,

Could you post the profiling output with HV 6? It should be a bit different, particularly regarding the classmate calls.

Any chance you could provide us with a Maven project allowing to reproduce the issue? It's really hard to provide feedback if we can't have a look at the setup and run the profiling by ourselves.

It shouldn't be too hard I think.

Thanks for your help.

--
Guillaume


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Wed Jun 07, 2017 6:55 am 
Newbie

Joined: Tue Jun 06, 2017 6:16 am
Posts: 4
gsmet wrote:
Any chance you could provide us with a Maven project allowing to reproduce the issue? It's really hard to provide feedback if we can't have a look at the setup and run the profiling by ourselves.

Sure. Below you can find the complete code and the pom.xml:

com.example.demo.controller.Controller
Code:
package com.example.demo.controller;

import java.security.InvalidParameterException;
import java.util.Set;

import javax.validation.ConstraintViolation;
import javax.validation.Valid;
import javax.validation.Validator;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import com.example.demo.model.Shop;

@RestController
@RequestMapping("/test")
public class Controller {
   private final Validator validator;

   public Controller() {
      validator = ValidatorFactorySingleton.getInstance().getValidator();
   }

   /**
    * No input-validation.
    *
    * @param shop
    * @return
    */
   @RequestMapping(path = "/no_validation", method = RequestMethod.POST)
   public ResponseEntity<Object> postNoValidation(@RequestBody Shop shop) {
      return new ResponseEntity<>(HttpStatus.CREATED);
   }

   /**
    * Validation using @Valid.
    *
    * @param shop
    * @return
    */
   @RequestMapping(path = "/spring", method = RequestMethod.POST)
   public ResponseEntity<Object> postSpringValidation(@RequestBody @Valid Shop shop) {
      return new ResponseEntity<>(HttpStatus.CREATED);
   }

   /**
    * Validation using the Validator.
    *
    * @param shop
    * @return
    */
   @RequestMapping(path = "/validator", method = RequestMethod.POST)
   public ResponseEntity<Object> postValidator(@RequestBody Shop shop) {
      final Set<ConstraintViolation<Shop>> violations = this.validator.validate(shop);

      if (!violations.isEmpty()) {
         throw new InvalidParameterException("invalid data");
      }

      return new ResponseEntity<>(HttpStatus.CREATED);
   }

   /**
    * Validation using our own method.
    *
    * @param shop
    * @return
    */
   @RequestMapping(path = "/own", method = RequestMethod.POST)
   public ResponseEntity<Object> postOwnValidation(@RequestBody Shop shop) {
      if (!shop.isValidData()) {
         throw new InvalidParameterException("invalid data");
      }

      return new ResponseEntity<>(HttpStatus.CREATED);
   }
}



com.example.demo.controller.ValidatorFactorySingleton
Code:
package com.example.demo.controller;

import javax.validation.Validation;
import javax.validation.ValidatorFactory;

public class ValidatorFactorySingleton {
   private static ValidatorFactory instance = null;

   private ValidatorFactorySingleton() {
   }

   public static ValidatorFactory getInstance() {
      if (instance == null) {
         instance = Validation.buildDefaultValidatorFactory();
      }

      return instance;
   }

}



com.example.demo.model.Article
Code:
package com.example.demo.model;

import javax.validation.constraints.NotNull;

import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

@Getter
@Setter
@NoArgsConstructor
public class Article {
   @NotNull
   private String id;

   public boolean isValidData() {
      if (id == null)
         return false;

      return true;
   }
}



com.example.demo.model.Shop
Code:
package com.example.demo.model;

import java.util.List;

import javax.validation.Valid;
import javax.validation.constraints.NotNull;

import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

@Getter
@Setter
@NoArgsConstructor
public class Shop {
   @NotNull
   private String id;

   @Valid
   @NotNull
   private List<Article> articles;

   public boolean isValidData() {
      if (id == null)
         return false;

      for (Article article : articles) {
         if (!article.isValidData()) {
            return false;
         }
      }

      return true;
   }
}



com.example.demo.PerformanceValidationApplication
Code:
package com.example.demo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class PerformanceValidationApplication {
   public static void main(String[] args) {
      SpringApplication.run(PerformanceValidationApplication.class, args);
   }
}



com.example.demo.controller.ControllerTest
Code:
package com.example.demo.controller;

import static org.assertj.core.api.Assertions.assertThat;

import java.util.ArrayList;
import java.util.List;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.context.ApplicationContext;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RequestCallback;
import org.springframework.web.client.ResponseExtractor;

import com.example.demo.model.Article;
import com.example.demo.model.Shop;
import com.fasterxml.jackson.databind.ObjectMapper;

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class ControllerTest {
   private static final int NUMBER_OF_RUNS_FOR_AVERAGE = 200;
   private static final int NUMBER_OF_ARTICLES_PER_SHOP = 20000;

   @Autowired
   private ApplicationContext appContext;

   @Autowired
   private ObjectMapper objectMapper;

   @Autowired
   private TestRestTemplate testRestTemplate;

   @Test
   public void contextLoads() throws Exception {
      assertThat(this.appContext).isNotNull();
   }

   @Test
   public void noValidation() {
      testEndpoint("/test/no_validation");
   }

   @Test
   public void springValidation() {
      testEndpoint("/test/spring");
   }

   @Test
   public void validator() {
      testEndpoint("/test/validator");
   }

   @Test
   public void ownValidation() {
      testEndpoint("/test/own");
   }

   private void testEndpoint(String url) {
      double runtime = getAverageRuntimeMillis(url, NUMBER_OF_RUNS_FOR_AVERAGE);
      System.out.println(url + " -> " + runtime + " ms");
   }

   private double getAverageRuntimeMillis(String url, int runs) {
      List<Long> runtimes = new ArrayList<>(runs);

      for (int i = 0; i < runs; i++) {
         long runtime = getRuntimeMillis(url);
         runtimes.add(runtime);
      }

      return runtimes.stream().mapToDouble(x -> x).average().getAsDouble();
   }

   private long getRuntimeMillis(String url) {
      long start = System.currentTimeMillis();
      this.executePost(url, createShop());
      long end = System.currentTimeMillis();

      return end - start;
   }

   private Shop createShop() {
      List<Article> articles = new ArrayList<>(NUMBER_OF_ARTICLES_PER_SHOP);

      for (int i = 0; i < NUMBER_OF_ARTICLES_PER_SHOP; i++) {
         Article article = new Article();
         article.setId("a" + i);
         articles.add(article);
      }

      Shop shop = new Shop();
      shop.setId("shop");
      shop.setArticles(articles);

      return shop;
   }

   private void executePost(String url, Shop shop) {
      assertThat(shop).isNotNull();

      final RequestCallback requestCallback = request -> {
         request.getHeaders().setContentType(MediaType.APPLICATION_JSON);

         request.getBody().write(objectMapper.writeValueAsBytes(shop));
      };

      final ResponseExtractor<?> responseExtractor = response -> {
         assertThat(response.getStatusCode()).isEqualTo(HttpStatus.CREATED);
         return null;
      };

      this.testRestTemplate.execute(url, HttpMethod.POST, requestCallback, responseExtractor);
   }
}



pom.xml
Code:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>

   <groupId>com.example</groupId>
   <artifactId>performance_validation</artifactId>
   <version>0.0.1-SNAPSHOT</version>
   <packaging>jar</packaging>

   <name>performance_validation</name>
   <description>Demo project for Spring Boot</description>

   <parent>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-parent</artifactId>
      <version>1.5.3.RELEASE</version>
      <relativePath /> <!-- lookup parent from repository -->
   </parent>

   <properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
      <java.version>1.8</java.version>
   </properties>

   <dependencies>
      <dependency>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-web</artifactId>
      </dependency>

      <dependency>
         <groupId>org.projectlombok</groupId>
         <artifactId>lombok</artifactId>
         <optional>true</optional>
      </dependency>

      <dependency>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-test</artifactId>
         <scope>test</scope>
      </dependency>
   </dependencies>

   <build>
      <plugins>
         <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
         </plugin>
      </plugins>
   </build>
</project>




gsmet wrote:
Could you post the profiling output with HV 6? It should be a bit different, particularly regarding the classmate calls.

I will come back to this


Top
 Profile  
 
 Post subject: Re: Performance of Hibernate Validator
PostPosted: Tue Sep 05, 2017 11:59 am 
Hibernate Team
Hibernate Team

Joined: Thu Jan 05, 2017 6:04 am
Posts: 13
Thanks for the benchmark. I added one related benchmark to our JMH tests.

I made a few improvements related to this issue in https://github.com/hibernate/hibernate- ... r/pull/845 but, to be clear, it doesn't improve your situation that much.

The main issue is that HV needs to keep track of the validated elements and the maintenance of the list of already processed elements is kinda heavy in your case. Unfortunately, there's nothing we can do here as not processing already processed elements is required by the spec.

Maybe we could provide an option specific to HV to not do it, but I'm unsure how it will be beneficial to the users as I expect they won't find it.

Will talk to Gunnar about that when he's back from vacation.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 11 posts ] 

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.