Kuesa Serenity Picking Example
Demonstrates the use of the KuesaSerenity API to import a glTF2 file and perform 3D mouse picking on objects from the scene.
Subclass Kuesa::Serenity::Window
We want to be able to react to mouse events. Therefore one option is to subclass the Window class so that we can override the mousePressEvent, mouseReleaseEvent and mouseMoveEvent.
We can register callbacks through the use of std::function to later specify the actions we want to perform whenever any of those events is received by the Window.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39 | class Window : public Kuesa::Serenity::Window
{
public:
Window() = default;
~Window() = default;
void setMousePressCallback(const std::function<bool(const KDGui::MousePressEvent *)> &callback) { m_mousePressCallback = callback; }
void setMouseReleaseCallback(const std::function<bool(const KDGui::MouseReleaseEvent *)> &callback) { m_mouseReleaseCallback = callback; }
void setMouseMoveCallback(const std::function<bool(const KDGui::MouseMoveEvent *)> &callback) { m_mouseMoveCallback = callback; }
private:
void mousePressEvent(KDGui::MousePressEvent *ev) override
{
if (m_mousePressCallback)
ev->setAccepted(m_mousePressCallback(ev));
else
ev->setAccepted(false);
}
void mouseReleaseEvent(KDGui::MouseReleaseEvent *ev) override
{
if (m_mouseReleaseCallback)
ev->setAccepted(m_mouseReleaseCallback(ev));
else
ev->setAccepted(false);
}
void mouseMoveEvent(KDGui::MouseMoveEvent *ev) override
{
if (m_mouseMoveCallback)
ev->setAccepted(m_mouseMoveCallback(ev));
else
ev->setAccepted(false);
}
std::function<bool(const KDGui::MousePressEvent *)> m_mousePressCallback;
std::function<bool(const KDGui::MouseReleaseEvent *)> m_mouseReleaseCallback;
std::function<bool(const KDGui::MouseMoveEvent *)> m_mouseMoveCallback;
};
|
Filename: picking-serenity/main.cpp
Creating the Application, Window and setting up KDGui
A KDGui::GuiApplication will be used to launch the application execution and even loop.
| KDGui::GuiApplication app;
|
Filename: picking-serenity/main.cpp
Next we instantiate our window and set its title.
| // Setup Window
Window w;
auto engine = std::make_unique<::Serenity::AspectEngine>();
w.visible.valueChanged().connect([&app](const bool &visible) {
if (visible == false)
app.quit();
});
w.title = KDBindings::makeBinding(makeTitle(w.width, w.height, engine->fps));
w.width = 1920;
w.height = 1080;
w.visible = true;
|
Filename: picking-serenity/main.cpp
We then set up KDGpu to specify which GraphicsAPI we want to be using and which underlying Graphics Device we want to be using.
| // Setup KDGpu
std::unique_ptr<KDGpu::GraphicsApi> api = std::make_unique<KDGpu::VulkanGraphicsApi>();
KDGpu::Instance instance = api->createInstance(KDGpu::InstanceOptions{
.applicationVersion = SERENITY_MAKE_API_VERSION(0, 1, 0, 0) });
const KDGpu::SurfaceOptions surfaceOptions = KDGpuKDGui::View::surfaceOptions(&w);
KDGpu::Surface surface = instance.createSurface(surfaceOptions);
KDGpu::AdapterAndDevice defaultDevice = instance.createDefaultDevice(surface);
KDGpu::Device device = std::move(defaultDevice.device);
|
Filename: picking-serenity/main.cpp
Creating the scene and importing a glTF2 File
We begin by adding Kuesa Layers to the [LayerManager ] so that meshes can be tagged with the appropriate layer mask based on their material properties upon importing.
This will be required to filter which Entities to render at the appropriate time.
| // Setup Kuesa Scene
::Serenity::LayerManager layerManager;
layerManager.addLayer(Kuesa::Serenity::Layers::ZFillLayerName);
layerManager.addLayer(Kuesa::Serenity::Layers::OpaqueLayerName);
layerManager.addLayer(Kuesa::Serenity::Layers::AlphaLayerName);
|
Filename: picking-serenity/main.cpp
Then we can proceed by actually importing a glTF 2 file, creating the scene content and populating the AssetCollections from the content of the imported scene file.
| Kuesa::Serenity::AssetCollections collections;
std::unique_ptr<::Serenity::Entity> root = createKuesaScene(&collections, &layerManager);
|
Filename: picking-serenity/main.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27 | std::unique_ptr<::Serenity::Entity> createKuesaScene(Kuesa::Serenity::AssetCollections *assetCollections,
::Serenity::LayerManager *layerManager)
{
Kuesa::Serenity::GLTF2Importer importer;
importer.assetCollections = assetCollections;
importer.layerManager = layerManager;
importer.source = String(SHARED_ASSETS_DIR "/models/picking/picking-scene.gltf");
// Insert the generated Kuesa Entities into rootEntity
auto rootEntity = std::make_unique<::Serenity::Entity>();
{
// Add a default light
::Serenity::Entity *e = rootEntity->createChildEntity<::Serenity::Entity>();
::Serenity::Light *l = e->createComponent<::Serenity::Light>();
l->type = ::Serenity::Light::Type::Point;
::Serenity::SrtTransform *t = e->createComponent<::Serenity::SrtTransform>();
t->translation = glm::vec3(0.0f, 40.0f, 0.0f);
}
for (std::unique_ptr<::Serenity::Entity> &sceneRoot : importer.sceneRoots()) {
// Take ownership
rootEntity->addChildEntity(std::move(sceneRoot));
}
return std::move(rootEntity);
}
|
Filename: picking-serenity/main.cpp
Once loaded, we can then retrieve the camera we will want to use to view the scene. We add a binding to update the aspectRatio of the camera lens whenever the window dimension change. Also, we instantiate a CameraController in order to be allow to move the camera around.
| // Retrieve Camera
::Serenity::Camera *camera = collections.cameras("Camera");
camera->lens()->aspectRatio = KDBindings::makeBinding(updateAspectRatio(w.width, w.height));
// Camera Controller
Kuesa::Serenity::CameraController *controller = w.createChild<Kuesa::Serenity::CameraController>();
controller->window = &w;
controller->camera = camera;
w.cameraController = controller;
|
Filename: picking-serenity/main.cpp
Setting up the Serenity Engine
The next step is about configuring the Serenity Engine. We first instantiate the aspects we want to use.
| // Setup Serenity Engine
auto renderAspect = engine->createAspect<::Serenity::RenderAspect>(std::move(device));
auto spatialAspect = engine->createAspect<::Serenity::SpatialAspect>();
auto logicAspect = engine->createAspect<::Serenity::LogicAspect>();
|
Filename: picking-serenity/main.cpp
Then we create the RenderAlgorithm that will be used by the RenderAspect.
| // Create Render Algo
std::unique_ptr<Serenity::ForwardAlgorithm> algo = createRenderAlgo(&layerManager, &w, instance, camera);
|
Filename: picking-serenity/main.cpp
We make use of the [ForwardAlgorithm ] render algorithm. It allows use to register RenderViews of one or more RenderPhases to control onto which RenderTarget we want to render onto and in how many steps.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78 | std::unique_ptr<::Serenity::ForwardAlgorithm> createRenderAlgo(::Serenity::LayerManager *layerManager,
Kuesa::Serenity::Window *w,
KDGpu::Instance &instance,
::Serenity::Camera *camera)
{
auto algo = std::make_unique<::Serenity::ForwardAlgorithm>();
algo->msaaSamples = Serenity::RenderAlgorithm::SamplesCount::Samples_4;
auto createOpaquePhase = [layerManager]() {
::Serenity::ForwardAlgorithm::RenderPhase opaquePhase{
layerManager->layerMask({ Kuesa::Serenity::Layers::OpaqueLayerName }),
::Serenity::ForwardAlgorithm::RenderPhase::Type::Opaque,
::Serenity::LayerFilterType::AcceptAny,
{},
true
};
auto depthState = std::make_shared<::Serenity::DepthStencilState>();
depthState->depthTestEnabled = true;
depthState->depthWritesEnabled = true;
depthState->depthCompareOperation = KDGpu::CompareOperation::Less;
opaquePhase.renderStates.setDepthStencilState(std::move(depthState));
auto rasterizerState = std::make_shared<::Serenity::PrimitiveRasterizerState>();
rasterizerState->cullMode = KDGpu::CullModeFlagBits::BackBit;
opaquePhase.renderStates.setPrimitiveRasterizerState(std::move(rasterizerState));
return opaquePhase;
};
auto createAlphaPhase = [layerManager]() {
::Serenity::ForwardAlgorithm::RenderPhase alphaPhase{
layerManager->layerMask({ Kuesa::Serenity::Layers::AlphaLayerName }),
::Serenity::ForwardAlgorithm::RenderPhase::Type::Alpha,
::Serenity::LayerFilterType::AcceptAll,
{},
true
};
auto depthState = std::make_shared<::Serenity::DepthStencilState>();
depthState->depthTestEnabled = true;
depthState->depthWritesEnabled = false;
depthState->depthCompareOperation = KDGpu::CompareOperation::LessOrEqual;
alphaPhase.renderStates.setDepthStencilState(std::move(depthState));
auto blendState = std::make_shared<::Serenity::ColorBlendState>();
::Serenity::ColorBlendState::AttachmentBlendState attachmentBlendState;
attachmentBlendState.format = KDGpu::Format::UNDEFINED;
attachmentBlendState.blending.blendingEnabled = true;
attachmentBlendState.blending.alpha.operation = KDGpu::BlendOperation::Add;
attachmentBlendState.blending.color.operation = KDGpu::BlendOperation::Add;
attachmentBlendState.blending.alpha.srcFactor = KDGpu::BlendFactor::SrcAlpha;
attachmentBlendState.blending.color.srcFactor = KDGpu::BlendFactor::SrcAlpha;
attachmentBlendState.blending.color.dstFactor = KDGpu::BlendFactor::OneMinusSrcAlpha;
attachmentBlendState.blending.alpha.dstFactor = KDGpu::BlendFactor::One;
blendState->attachmentBlendStates = { attachmentBlendState };
alphaPhase.renderStates.setColorBlendState(std::move(blendState));
return alphaPhase;
};
// Specify RT
algo->renderTargetRefs = { Serenity::RenderTargetRef::fromWindow(w, instance) };
// Specify which RenderTarget to render to, which render phases and which camera to use
algo->renderViews = {
Serenity::ForwardAlgorithm::RenderView(0,
{ createOpaquePhase(),
createAlphaPhase() },
nullptr,
camera)
};
return algo;
}
|
Filename: picking-serenity/main.cpp
Lastly, we set the RenderAlgorithm on the RenderAspect, set the root entity on the engine and start it.
| renderAspect->setRenderAlgorithm(std::move(algo));
engine->setRootEntity(std::move(root));
engine->running = true;
|
Filename: picking-serenity/main.cpp
Handling Picking
To be able to perform picking against entities in the scene, we first need to assign them BoundingVolumes.
| // To be able to perform picking, we need to add Bounding Volumes to the Entities we are trying to select
addBoundingVolumesToPickableEntities(&collections);
|
Filename: picking-serenity/main.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25 | const std::array<std::string, 4> pickableEntitiesNames = {
StringLiteral("Cube"),
StringLiteral("Torus"),
StringLiteral("Cylinder"),
StringLiteral("Sphere"),
};
void addBoundingVolumesToPickableEntities(Kuesa::Serenity::AssetCollections *collections)
{
for (const auto &entityName : pickableEntitiesNames) {
::Serenity::Entity *e = collections->entities(entityName);
if (!e)
continue;
const auto meshRenderers = e->components<::Serenity::MeshRenderer>();
// Add a BoundingVolume for each MeshRenderer so that the Entity can be used for ray casting intersection tests
for (auto meshRenderer : meshRenderers) {
::Serenity::TriangleBoundingVolume *bv = e->createComponent<::Serenity::TriangleBoundingVolume>();
bv->meshRenderer = meshRenderer;
bv->cacheTriangles = true;
bv->cullBackFaces = true;
}
}
}
|
Filename: picking-serenity/main.cpp
We want to be able to change the size of the Entity on which we pressed. To make that as straightforward as possible, we create a currentlyPressedEntity property. Then we rely on the valueAboutToChange signal to register a callback that will take care of restoring the scale of the previously pressed entity and updating the newly selected one.
1
2
3
4
5
6
7
8
9
10
11
12
13 | KDBindings::Property<::Serenity::Entity *> currentlyPressedEntity{ nullptr };
currentlyPressedEntity.valueAboutToChange().connect([&](::Serenity::Entity *currentValue, ::Serenity::Entity *newValue) {
if (currentValue != nullptr) {
::Serenity::SrtTransform *transform = currentValue->component<::Serenity::SrtTransform>();
if (transform)
transform->scale = glm::vec3(1.0);
}
if (newValue != nullptr) {
::Serenity::SrtTransform *transform = newValue->component<::Serenity::SrtTransform>();
if (transform)
transform->scale = glm::vec3(1.5);
}
});
|
Filename: picking-serenity/main.cpp
To update the currentlyPressedEntity, we rely on registering callbacks on the pressed and released event handlers of our Window.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16 | w.setMousePressCallback([&](const KDGui::MousePressEvent *ev) {
::Serenity::Entity *hitEntity = issueScreenCast(&w, spatialAspect, camera, glm::vec2(ev->xPos(), ev->yPos()));
if (hitEntity) {
currentlyPressedEntity = hitEntity;
return true;
}
return false;
});
w.setMouseReleaseCallback([&](const KDGui::MouseReleaseEvent *ev) {
if (currentlyPressedEntity()) {
currentlyPressedEntity = nullptr;
return true;
}
return false;
});
|
Filename: picking-serenity/main.cpp
The press callbacks triggers a ray casting against the scene to be able to detect which object lie under the mouse.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26 | ::Serenity::Entity *issueScreenCast(Window *w,
::Serenity::SpatialAspect *spatialAspect,
::Serenity::Camera *camera,
glm::vec2 mousePos)
{
// Perform ray cast
std::vector<::Serenity::SpatialAspect::Hit> hits = spatialAspect->screenCast(mousePos,
glm::vec4(0.0f, 0.0f, w->width(), w->height()),
camera->viewMatrix(),
camera->lens()->projectionMatrix());
// Sort by distance
std::sort(hits.begin(), hits.end(), [](const auto &a, const auto &b) { return a.distance < b.distance; });
// Select closest hit whose Entity matches one of the expected pickable Entities name
for (const ::Serenity::SpatialAspect::Hit &hit : hits) {
auto it = std::find_if(std::begin(pickableEntitiesNames),
std::end(pickableEntitiesNames), [&hit](const std::string &name) {
return hit.entity->objectName() == name;
});
if (it != std::end(pickableEntitiesNames)) {
return hit.entity;
}
}
return nullptr;
}
|
Filename: picking-serenity/main.cpp
Similarly, to be able to change the appearance of the objects being hovered, we defined a currentlyHoveredEntity property. Then we rely on the valueAboutToChange signal to register a callback that will take care changing the color of the object under the cursor.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23 | KDBindings::Property<::Serenity::Entity *> currentlyHoveredEntity{ nullptr };
currentlyHoveredEntity.valueAboutToChange().connect([&](::Serenity::Entity *currentValue, ::Serenity::Entity *newValue) {
static Color previousBaseColorFactor = {};
if (currentValue != nullptr) {
const auto meshRenderers = currentValue->components<::Serenity::MeshRenderer>();
for (auto meshRenderer : meshRenderers) {
auto *m = static_cast<Kuesa::Serenity::GLTF2Material *>(meshRenderer->material());
if (m)
m->baseColorFactor = previousBaseColorFactor;
}
}
if (newValue != nullptr) {
const auto meshRenderers = newValue->components<::Serenity::MeshRenderer>();
for (auto meshRenderer : meshRenderers) {
auto *m = static_cast<Kuesa::Serenity::GLTF2Material *>(meshRenderer->material());
if (m) {
previousBaseColorFactor = m->baseColorFactor();
m->baseColorFactor = { 1.0, 0.0, 0.0, 1.0 };
}
}
}
});
|
Filename: picking-serenity/main.cpp
To update the currentlyHoveredEntity, we rely on registering callbacks on the move handler of our Window. The callbacks triggers a raycast and updates the currentlyHoveredEntity.
| w.setMouseMoveCallback([&](const KDGui::MouseMoveEvent *ev) {
::Serenity::Entity *hitEntity = issueScreenCast(&w, spatialAspect, camera, glm::vec2(ev->xPos(), ev->yPos()));
if (hitEntity || currentlyHoveredEntity()) {
currentlyHoveredEntity = hitEntity;
return true;
}
return false;
});
|
Filename: picking-serenity/main.cpp
Updated on 2023-07-03 at 11:02:17 +0000