Can not persist Business Data to Repo after changing data type

1

Hi all

I'm using Ivy 6.3 and use ivy.repo api to persist data. When i persits data as this JSON:

"qualificationMessageResults": [
        {
            "code": "amount",
            "severity": "ORANGE",
            "message": {
                "descriptor": {
                    "content": "/ch/axonivy/fintech/soba/Message/PreQualification/Amount/amountBetweenMinAmountAndMaxAmountForReplacement",
                    "type": "URI"
                },
                "paramDescriptors": [
                    {
                        "descriptor": {
                            "content": "500000",
                            "type": "PLAIN_TEXT",
                            "placeHolder": "{minAmount}"
                        },
                        "paramDescriptors": []
                    }
                ]
            }
        }
    ]
},

The api say:

 [errorId=15BCE2ED377AD1E0, request=HTTP POST ch/axonivy/fintech/soba/showcase/DemoSobaPersistence/DemoSobaPersistenceProcess.mod/15A1694DAA1C0A9F(9907.9907.17.0), session=1, task=9907, application=102, requestId=4093, executionContext=1, pmv=Portal$soba_test$2, hd=ch.axonivy.fintech.soba.showcase.DemoSobaPersistence, processElement=15A168DC648B6158-f9]
Caused by: ch.ivyteam.ivy.persistence.PersistencyException: Could not create BusinessData.
    at ch.ivyteam.ivy.business.data.store.internal.ElasticSystemDbPersistence.execute(ElasticSystemDbPersistence.java:115)
    at ch.ivyteam.ivy.business.data.store.internal.ElasticSystemDbPersistence.execute(ElasticSystemDbPersistence.java:104)
    at ch.ivyteam.ivy.business.data.store.internal.ElasticSystemDbPersistence.create(ElasticSystemDbPersistence.java:89)
    at ch.ivyteam.ivy.business.data.store.internal.BusinessDataRepositoryImpl.persist(BusinessDataRepositoryImpl.java:112)
    at ch.ivyteam.ivy.business.data.store.internal.BusinessDataRepositoryImpl.save(BusinessDataRepositoryImpl.java:96)
    at ch.axonivy.fintech.standard.dossier.persistence.BaseDossierPersistence.persist(BaseDossierPersistence.java:35)
    at ch.axonivy.fintech.soba.dossier.service.SobaDossierService.persist(SobaDossierService.java:155)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at ch.ivyteam.ivy.scripting.internal.types.IvyJavaMethod.invokeImpl(IvyJavaMethod.java:73)
    ... 208 more
[errorId=15BCE2ED377AD1E0, request=HTTP POST ch/axonivy/fintech/soba/showcase/DemoSobaPersistence/DemoSobaPersistenceProcess.mod/15A1694DAA1C0A9F(9907.9907.17.0), session=1, task=9907, application=102, requestId=4093, executionContext=1, pmv=Portal$soba_test$2, hd=ch.axonivy.fintech.soba.showcase.DemoSobaPersistence, processElement=15A168DC648B6158-f9]
Caused by: ch.ivyteam.ivy.business.data.store.search.internal.elasticsearch.ElasticsearchException: Elasticsearch update index of document failed because of: {"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse [onlineMortgageProgress.qualificationMessageResults.message]"}],"type":"mapper_parsing_exception","reason":"failed to parse [onlineMortgageProgress.qualificationMessageResults.message]","caused_by":{"type":"illegal_argument_exception","reason":"unknown property [descriptor]"}}
    at ch.ivyteam.ivy.business.data.store.search.internal.elasticsearch.JestOperation.checkStatusAndThrow(JestOperation.java:61)
    at ch.ivyteam.ivy.business.data.store.search.internal.elasticsearch.JestOperation.execute(JestOperation.java:39)
    at ch.ivyteam.ivy.business.data.store.search.internal.elasticsearch.JestIndexSynchronizer.execute(JestIndexSynchronizer.java:165)
    at ch.ivyteam.ivy.business.data.store.search.internal.elasticsearch.JestIndexSynchronizer.updateIndex(JestIndexSynchronizer.java:64)
    at ch.ivyteam.ivy.business.data.store.search.internal.ElasticBusinessDataSearchIndex.update(ElasticBusinessDataSearchIndex.java:30)
    at ch.ivyteam.ivy.business.data.store.internal.ElasticSystemDbPersistence.lambda$1(ElasticSystemDbPersistence.java:96)
    at ch.ivyteam.ivy.persistence.base.AbstractPersistencyService.execute(AbstractPersistencyService.java:177)
    at ch.ivyteam.ivy.persistence.base.AbstractPersistencyService.execute(AbstractPersistencyService.java:240)
    at ch.ivyteam.ivy.persistence.base.ClassPersistencyService.execute(ClassPersistencyService.java:694)
    at

My object QualificationMessageResult has list of Object A, and object A has two properties: a object descriptor and a list of Object A (looped Object) with name: paramDescriptors.

Can anyone suggest me something?

Thanks

asked 03.05.2017 at 09:41

trungdv's gravatar image

trungdv
(suspended)
accept rate: 52%

edited 04.07.2017 at 08:59

SupportIvyTeam's gravatar image

SupportIvyTeam ♦♦
1.4k●102●118●122

i have found the root cause. it's because on new version we just changed the type of property qualificationMessageResults.message from String to an object type, so it lead to issue that when we deploy this version to server, the old business data (where message is still String type) can not be index anymore. And it throw exception unknow property descriptor.

Problematic is that ElasticSearch already determined the type of this field as string. I heard elasticsearch has way to reindex, does ivy support it?

(25.05.2017 at 22:01) trungdv trungdv's gravatar image

Currently there is no API in the axon.ivy core that would trigger an index-recreation or schema update on the ElasticSearch server. But you could basically flush the whole ElasticSearch index and trigger the axon.ivy engine to synchronize its data again with the ElasticSearch index. The API to do so is here: ch.ivyteam.ivy.business.data.store.restricted.IBusinessDataManager.startIndexRecreation(). There is also a built in mechanism on axon.ivy engine startup that synchronizes the indexes if the ElasticSeach index is not up to date.

(26.05.2017 at 05:19) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image
One Answer:
active answersoldestnewestmost voted
1

You do not run into this problem if you just introduce a new attribute for the new type instead of re-using the existing attribute that already contains other data. Business Data is explicitly designed to allow simple data migration. So it does not throw any errors for unknown fields.

However, there is a work around

If you just wrapped the String in a complex object you could work around the problem by influencing the JSON serialization. With Jackson you could for instance store the new complex object also as string and read it back to a complex object. And of course this also allows you to handle legacy serialized objects.

The rough concept is described here: http://developer.axonivy.com/doc/latest/DesignerGuideHtml/ivy.datamodeling.html#ivy.businessdata.serialization.module

A sample implementation could do the following. Here we serialize a complex Person object as a raw String.

FIRST: Write your own serializer module:

package com.axonivy;

import java.io.IOException;

import org.apache.commons.lang3.StringUtils;

import ch.ivyteam.ivy.environment.Ivy;

import com.axonivy.sample.Person;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.deser.std.StdScalarDeserializer;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.ser.std.StdScalarSerializer;

public class CustomBusinessDataModule extends SimpleModule {

    private static final long serialVersionUID = 1L;
    private static final String SEPARATOR = "|";

    public CustomBusinessDataModule()
    {
        Ivy.log().info("registered custom module!");
        addSerializer(Person.class, new CustomPersonSerializer());
        addDeserializer(Person.class, new CustomPersonDeSerializer());
    }

    @SuppressWarnings("serial")
    private static class CustomPersonSerializer extends StdScalarSerializer<Person>
    {

        public CustomPersonSerializer() {
            super(Person.class);
        }

        @Override
        public void serialize(Person value, JsonGenerator gen,
                SerializerProvider provider) throws IOException {
            gen.writeString(value.getFirstname()+SEPARATOR+value.getLastname());
        }
    }

    @SuppressWarnings("serial")
    private static class CustomPersonDeSerializer extends StdScalarDeserializer<Person>
    {
        public CustomPersonDeSerializer()
        {
            super(Person.class);
        }

        @Override
        public Person deserialize(JsonParser p, DeserializationContext ctxt)
                throws IOException, JsonProcessingException {
            String value = p.readValueAs(String.class);
            if (!StringUtils.contains(value, SEPARATOR))
            {
                // old version.... -> take special care!
                return new Person();
            }

            String[] parts = StringUtils.split(value, SEPARATOR);
            Person person = new Person();
            person.setFirstname(parts[0]);
            person.setLastname(parts[1]);
            return person;
        }
    }

}

SECOND: register it via service SPI:

alt text

Here you see the custom serializer in action:

businessDataSerializer_66.iar alt text

link

answered 04.07.2017 at 08:29

Reguel%20Wermelinger's gravatar image

Reguel Werme... ♦♦
9.4k●3●19●58
accept rate: 70%

edited 07.03.2018 at 10:22

Alex%20Suter's gravatar image

Alex Suter ♦♦
3.1k●12●22●47

This demo is just for illustrative purposes. In a real world scenario you should pay special care that the serializer / deserializer implementation is rock solid. For simple demonstration I've just used a simple field separator here...

(04.07.2017 at 08:50) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

learn more about Markdown

Tags:

×7

Asked: 03.05.2017 at 09:41

Seen: 3,095 times

Last updated: 07.03.2018 at 10:22

Related questions

Can not use ivy.repo in ResfulAPI

There is a 1-second latency between ivy.repo.save(object) and ivy.repo.search(Object.class)

Can we persist Elastic Search data to Business Data table?

Can not connect to Elasticsearch server

How to persist fields in ivy repo but not index in elastic?

How to upgrade elasticsearch engine on designer?