Intel Realsense SDK OpenVINO wrapper for the D435 depth camera
Hi everyone,
I would like to start using the OpenVINO wrapper for my project, which I am developing in Visual C++ with the Intel Realsense D435 depth camera. In particular, I need it for a face detection application.
I've seen the examples on the librealsense GitHub page but it's not completely clear to me how to import the OpenVINO models into my Visual Studio solution. Can someone help me understand how to get started with the wrapper and how to use it for a real time detection application on the color frames acquired by the depth camera?
Thank you very much
-
Hi Gbc-gg,
Sorry for the delay in replying to you. You can refer to this installation step to Install OpenVINO: https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_windows.html.
To debug or run the samples on Windows in Microsoft Visual Studio, make sure you have properly configured Debugging environment settings for the Debug and Release configurations. Set correct paths to the OpenCV libraries, and debug and release versions of the Inference Engine libraries. For example, for the Debug configuration, go to the project's Configuration Properties to the Debugging category and set thePATHvariable in the Environment field to the following:PATH=<INSTALL_DIR>\deployment_tools\inference_engine\bin\intel64\Debug;<INSTALL_DIR>\opencv\bin;%PATH%
where
<INSTALL_DIR>is the directory in which the OpenVINO toolkit is installed.
You can find more information in this https://docs.openvinotoolkit.org/2021.4/openvino_docs_IE_DG_Samples_Overview.html page.
Regards,
Aznie
Intel RealSense Customer Support -
Hi Aznie,
Thank you very much for your response, I've now managed to successfully compile and run my application in Debug mode with OpenViNO. However, I tried to refer to this sample from the librealsense/openvino documentation and, when I try to run it in Release configuration, it crashes at the following line:
faceDetector.load_into(engine, device_name);
In particular, it crashes when trying to load the network with LoadNetwork (last line of the load_into function from the base-detection.cpp file).
Am I missing something in the Release configuration? The
PATH=<INSTALL_DIR>\deployment_tools\inference_engine\bin\intel64\Release;
PATH variable in the Environment field is already set.
Hoping to receive your feedback,
Regards
-
Hi Gbc-gg,
To run in the release mode, I would recommend you to try installing/build the OpenVINO toolkit from the open-source repo: https://github.com/openvinotoolkit/openvino/wiki/BuildingForWindows and see if the same issue arises.
Regards,
Aznie
Intel RealSense Customer Support
-
Hi Aznie,
I see that the OpenViNO version in the link you sent me is the lastest (2021.4.1), which only supports Visual Studio 2019. However, I'm currently using Visual Studio 2017, and I'm using the OpenViNO toolkit provided by the Realsense SDK (which refers to the version 2019_R3).
At this point, I'm starting to get a bit confused, so I'm asking you: do I necessarily need to use version 2019_R3, or is there any particular version I should use in order to run the sample?
PS: I would also like to run this demo in order to estimate the head pose in real time from the Realsense depth camera.
Thank you,
Regards
-
UPDATE: I managed to build the VS solution with the OpenViNO toolkit 2020.1 built from source code (I couldn't figure out how to build 2019_R3 with cmake). However, when trying to load the .xml model, the app crashes and produces the following log:
[INFO] Loading object detection model to the CPU device
[INFO] Loading object detection model from: models/face-detection-retail-0004/face-detection-retail-0004.xml
[ERROR] Failed to load face detector model files:
Cannot create ShapeOf layer fc7_mbox_priorbox/0_port id:192Any ideas on why this happens?
PS: to successfully build the solution, I had to comment out some parts (bold parts of the following snippet) of code in the file win_shared_object_loader.h:
// Copyright (C) 2018-2020 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
/**
* @brief WINAPI compatible loader for a shared object
*
* @file win_shared_object_loader.h
*/
#pragma once
#include "ie_api.h"
#include "details/ie_exception.hpp"
#include "details/os/os_filesystem.hpp"
// Avoidance of Windows.h to include winsock library.
#define _WINSOCKAPI_
// Avoidance of Windows.h to define min/max.
#ifndef NOMINMAX
#define NOMINMAX
#endif
#include <direct.h>
#include <windows.h>
namespace InferenceEngine {
namespace details {
/**
* @brief This class provides an OS shared module abstraction
*/
class SharedObjectLoader {
private:
HMODULE shared_object;
void ExcludeCurrentDirectory() {
// Exclude current directory from DLL search path process wise.
// If application specific path was configured before then
// current directory is alread excluded.
// GetDLLDirectory does not distinguish if aplication specific
// path was set to "" or NULL so reset it to "" to keep
// aplication safe.
if (GetDllDirectory(0, NULL) <= 1) {
SetDllDirectory(TEXT(""));
}
}
public:
/**
* @brief Loads a library with the name specified. The library is loaded according to the
* WinAPI LoadLibrary rules
* @param pluginName Full or relative path to the plugin library
*/
explicit SharedObjectLoader(LPCWSTR pluginName) {
ExcludeCurrentDirectory();
shared_object = LoadLibraryW(pluginName);
if (!shared_object) {
char cwd[1024];
THROW_IE_EXCEPTION << "Cannot load library '" << details::wStringtoMBCSstringChar(std::wstring(pluginName)) << "': " << GetLastError()
<< " from cwd: " << _getcwd(cwd, sizeof(cwd));
}
}
explicit SharedObjectLoader(LPCSTR pluginName) {
ExcludeCurrentDirectory();
shared_object = LoadLibrary(pluginName);
if (!shared_object) {
char cwd[1024];
THROW_IE_EXCEPTION << "Cannot load library '" << pluginName << "': " << GetLastError()
<< " from cwd: " << _getcwd(cwd, sizeof(cwd));
}
}
~SharedObjectLoader() {
FreeLibrary(shared_object);
}
/**
* @brief Searches for a function symbol in the loaded module
* @param symbolName Name of function to find
* @return A pointer to the function if found
* @throws InferenceEngineException if the function is not found
*/
void* get_symbol(const char* symbolName) const {
if (!shared_object) {
THROW_IE_EXCEPTION << "Cannot get '" << symbolName << "' content from unknown library!";
}
auto procAddr = reinterpret_cast<void*>(GetProcAddress(shared_object, symbolName));
if (procAddr == nullptr)
THROW_IE_EXCEPTION << "GetProcAddress cannot locate method '" << symbolName << "': " << GetLastError();
return procAddr;
}
};
} // namespace details
} // namespace InferenceEngineThese are the errors I was getting:
'SetDllDirectory': identifier not found (compiling source file MainPage.cpp)
'GetDllDirectory': identifier not found (compiling source file MainPage.cpp)
'HMODULE LoadLibraryW(LPCWSTR)': cannot convert argument 1 from 'LPCSTR' to 'LPCWSTR' (compiling source file MainPage.cpp) -
UPDATE 2: I get these errors while trying to run the application with OpenViNO 2021.1:
[INFO] Loading object detection model to the CPU device
[INFO] Loading object detection model from: models/face-detection-retail-0004/face-detection-retail-0004.xml
[ERROR] Failed to load face detector model files:
invalid stoi argument
[INFO] Loading Head Pose model to the CPU device
[INFO] Loading network files for Head Pose Estimation network
[ERROR] Failed to load head pose estimator model files:
Cannot create Gather layer angle_p_fc/weights_transpose/Shape/Gather id:138 from unsupported opset: opset7
C:\j\workspace\private-ci\ie\build-windows-vs2019@2\b\repos\openvino\inference-engine\src\readers\ir_reader\ie_ir_parser.cpp:594 -
Hi Gbc-gg,
The issue is related to the OpenVINO toolkit. I would advise you to open a ticket to OpenVINO Community for further help.
Regards,
Aznie
Intel RealSense Customer Support
Please sign in to leave a comment.
Comments
8 comments