Browsing by Author "Jacob, Christian J."

Now showing 1 - 8 of 8
  • Thumbnail Image
    ItemOpen Access
    An Application of Calculated Consonance in Computer-Assisted Microtonal Music
    (2013-12-23) Burleigh, Ian George; Jacob, Christian J.; Eagle, David Malcolm
    Harmony (the audible result of varied combinations of simultaneously sounding tones) ought to, for the most part, sound pleasing to the ear. The result depends, among other factors, on a proper choice of the pitches for the tones that form harmonious chords, and on their correct intonation during musical performance. This thesis proposes a computational method for calculation of relative consonance among groups of tones, and its possible practical applications in machine-assisted arrangement of tones, namely the choice of tone pitches and their microtonal adjustment. The consonance of tone groups is calculated using a model that is based on the physiological theory of tone consonance that was published by Hermann Helmholtz in the middle of the 19th century. Given a group of tones that have fixed pitches, changes in the aggregate dissonance caused by adding another “probe” tone of a variable pitch can be represented as a “dissonance landscape”. Local minima in the “height” of the landscape correspond to local minima of the aggregate dissonance as a function of the pitch of the probe tone. Finding a local dissonance minimum simulates the actions of a musician who is “tuning by ear”. The set of all local minima within a given pitch range is a collection of potentially good pitch choices from which a composer (a human, or an algorithmic process) can fashion melodies that sound in harmony with the fixed tones. Several practical examples, realized in an experimental software, demonstrate applications of the method for: 1) computer-assisted microtonal tone arrangement (music composition), 2) algorithmic (machine-generated) music, and 3) musical interplay between a human and a machine. The just intonation aspect of the tuning method naturally leads to more than twelve, potentially to many, pitches in an octave. Without some restrictions that limit the complexity of the process, handling of so many possibilities by a human composer and their precise rendition as sound by a performing musician would be very difficult. Restricting the continuum of possible pitches to the discrete 53-division of the octave, and employing machine-assistance in their arrangement and in sound synthesis make applications of the method feasible.
  • Thumbnail Image
    ItemOpen Access
    Balanced Multiresolution in Multilevel Focus+Context Visualization
    (2018-08-22) Hasan, Mahmudul; Samavati, Faramarz; Costa Sousa, Mário; Mudur, Sudhir Pandurang; Gavrilova, Marina L.; Jacob, Christian J.; Katz, Larry
    Given a set of symmetric/antisymmetric filter vectors containing only regular multiresolution filters, the method we present in this thesis can establish a balanced multiresolution (BMR) scheme for images, allowing their balanced decomposition and subsequent perfect reconstruction without the use of any extraordinary boundary filters. We define balanced multiresolution such that it allows balanced decomposition i.e. decomposition of a high-resolution image into a low-resolution image and corresponding details of equal size. Several applications of such a decomposition result in a balanced wavelet transform (BWT) that makes on-demand reconstruction of regions of interest (ROIs) efficient in both computational load and implementation aspects. We find such decomposition and perfect reconstruction based on an appropriate combination of symmetric/antisymmetric extensions near the image and detail boundaries. In our method, exploiting such extensions correlates to performing sample (pixel/voxel) split operations. We demonstrate our general approach for some commonly used symmetric/antisymmetric multiresolution filters. We also show the application of such a balanced multiresolution scheme in constructing an interactive multilevel focus+context visualization framework for the navigation and exploration of large-scale 2D and 3D images. Typically, the given filters are floating-point values, so our BWTs reversibly map integers to floating-point i.e. real values. We extend our balanced multiresolution framework further to construct reversible integer-to-integer BWTs from a given symmetric/antisymmetric decomposition filter vector of width less or equal to four. In our approach, we adjust the linear combination of fine samples suggested by the given decomposition vector using optimal sample split operations in combination with a rounding operation. Such adjustments translate an affine integer combination of fine samples to obtain an integer coarse sample, which closely approximates the floating-point coarse sample suggested by the given decomposition filter vector. The associated translation vectors give us the detail samples. Furthermore, when necessary, we construct every other detail sample differently in order to ensure local perfect reconstruction. Compared to their integer-to-real counterparts, the resulting reversible integer-to-integer BWTs occupy less memory, offer better compressibility, and do not require sample quantization for rendering purposes.
  • Thumbnail Image
    ItemOpen Access
    Lucida: Enhancing the creation of photography with an Augmented Intelligence Digital Photography Agent
    (2018-10-22) Wrobleski, Brad; Gadbois, Denis; Jacob, Christian J.; Leblanc, Jean-René
    This thesis explores the research, design and development of a mobile intelligent agent to assist photographers. This work explores the integration of Augmented Reality (AR), Affective Computing (AC), Natural Language Processing (NLP), Computer Vision (CV), and more specifically, Machine Learning (ML) to make possible, the development of a mobile, sympathetic, ambient (virtual), augmented intelligence application (Agent) for assisting photographers. For this research, a prototype agent was designed (Lucida) and developed to assist photographers to enhance the understanding and creation of photographic metering and composition. Learning the art of photography is complicated because of the technical complexity of the camera, the limitations of the user to see photographically, and the lack of real time instruction, feedback and emotive support. A study was completed to measure the effect of how an artificially intelligent agent could communicate photography information. The study explored the communication and pedagogical patterns between human instructor and student and the disparity between human ability and the camera [71]. It analyzed how to transmute the communication and pedagogical values of the human to human experience to the digital milieu using Unsupervised Learning. This research specifically explored how an agent could be trained using the values and attributes of the act of photography and mimic the relationship with an instructor using artificial intelligence. This research focuses on designing and training the agent to find and identify an objective determination of tones in a photography scene (quantitative) and the subjective identification of composition possibilities (intuitive). This thesis illuminates the effectiveness and cogency of Agent based instruction and communication. Further, it demonstrates that a mobile, semantic, sympathetic, augmented, ambient agent can ameliorate the practice of photography in real time, on location by acting as the virtual intelligence to guide photographers. It shows that the integration of specific technologies, and design, produces an effective architecture for the creation of intelligent agent-based assistance.
  • Thumbnail Image
    ItemOpen Access
    Modelling Natural Phenomenon with Reaction-Diffusion
    (2020-01-22) Ringham, Lee; Prusinkiewicz, Przemysław; Alim, Usman R.; Jacob, Christian J.
    Procedural methods provide an algorithmic way to produce textures for use in computer graphics. One such method, reaction-diffusion, is a powerful mathematical approach that describes natural pattern formation in terms of chemicals known as morphogens. This thesis describes LRDS, an environment for authoring reaction-diffusion models directly on arbitrary surfaces. Morphogens, their behaviours, and the domain in which they reside can be quickly and easily defined. By performing computation on the GPU, the pattern forming simulation can be interacted with in real-time, facilitating productivity and experimentation. Four case studies are presented. The first is a simulation of ladybug pigmentation patterns. The second is a simulation of pigmentation patterns seen on the body of snakes. The third study looks at flower petal pattern modelling. Lastly, a biologically-motivated model of the autoimmune disease psoriasis is presented.
  • Thumbnail Image
    ItemOpen Access
    Mountain Pine Beetle Detection using IoT
    (2020-01-06) Rochester, Edward; Ghaderi, Majid; Reid, Mary L.; Jacob, Christian J.
    The large-scale damage caused by outbreaks of forest pests has significant impact on both the ecosystem and forestry industry. Currently, pest outbreaks are monitored using field, aerial and remote sensing surveys. These methods, however, only provide partial spatial coverage and can detect outbreaks only after they have substantially progressed across wide geographic areas. The goal of this thesis is to build and evaluate an IoT system for real-time Mountain Pine Beetle (MPB) infestation detection using bioacoustic recognition via deep learning techniques. First, we present the design of the IoT system and describe its various hardware and software components. We built our IoT device using LoRa communication technology and DIY components. We use a modified version of an adaptive differential pulse-code modulation for sample encoding to reduce the size of the recorded samples and prepare them for transmission. As a result, 15 packets are required to transmit a single, compressed half-a-second sample. The transmitted samples are then received at a gateway and forwarded to a client-server application for decoding, storing, analysis and visualization. Second, we analyze the MPB bioacoustic characteristics in indoor and outdoor scenarios with the use of live beetles. The samples collected using the designed IoT system hardware were manually analyzed and labeled for the convolutional neural network (CNN) training and testing. To predict the presence of MPB with trees, we evaluate a set of state-of-the-art (SOTA) convolutional neural networks (CNN) architectures on recorded samples and live-measurements. To use CNN we transform the recorded bioacoustic samples into spectrograms. Our evaluation results show that with low-resolution samples provided by the low-power, low data rate IoT devices, the SOTA CNN (specifically, ResNet152) can achieve up to 77% accuracy. We observe that even in the outdoor environment the model still achieves as high as 68% accuracy. We conclude by proposing several approaches for improving the proposed system's performance.
  • Thumbnail Image
    ItemOpen Access
    Multidimensional Projection Visualization: Control-points Selection and Inverse Projection Exploration
    (2016) Portes dos Santos Amorim, Elisa; Costa Sousa, Mário; Samavati, Faramarz; Gavrilova, Marina; Jacob, Christian J.; Rios, Cristian; Esperanca, Claudio
    The task of interpreting multidimensional data is as important as it is challenging. The importance comes from the fact that virtually every data worth analyzing is multidimensional, while the challenge comes from the very nature of these data sets, as the multiple features describing each instance can quickly overwhelm our visual perception system, thus making it difficult to observe meaningful information. Visualization techniques play an essential role in simplifying this task, by preprocessing the data to extract critical features and displaying them effectively, by using visual metaphors that can be easily understood. Multidimensional Projection (MP) is one of such techniques, whose fundamental goal is to present an overview of the data distribution in the form of a 2D scatterplot graph. It does so by reducing the dimensionality of the dataset in such a way that distances are preserved as much as possible. MP approaches, along with most visualizations, are shifting from a static display to a more interactive one, allowing human intervention to modify the layout and facilitate exploration and understanding of the data. In this thesis, I present contributions that specifically relate to interactive aspects of multidimensional projection. First, I propose a computational framework and methodology for control points selection. Control points are a particular set of projected points used to steer and rearrange the projection layout. I demonstrate the proposed method can improve the projection quality while requiring only a small amount of control points. Second, I introduce inverse projection, a novel paradigm to create multidimensional points exclusively through 2D interactions. The projection space is transformed into a canvas, where new points can be added. These new points are then mapped into the original multidimensional space, i.e., they become unique multidimensional instances themselves. Lastly, I present the usability of the inverse projection framework in two demonstration examples. (1) A parameter exploration prototype system for optimization with multiple minima. (2) A face-synthesis application, where new face models are generated on the fly.
  • Thumbnail Image
    ItemOpen Access
    The Task and Workslip Scheme for Personal Agent Systems
    (2019-05-29) Kendon, Tyson James; Denzinger, Jörg; Boyd, Jeffrey Edwin; Jacob, Christian J.; Finn, Patrick; Flores, Roberto A.
    The Task and Workslip Scheme for Personal Agent Systems is a collection of concepts enabling the intelligent communication of small agents, and a set of philosophical guidelines to aid in the design of intelligent systems to assist people in day-to-day tasks. The intention behind the scheme is to empower people to take control of their own computing power to achieve the kind of automatic personal assistance that otherwise can only be achieved with corporate computing. The scheme addresses the issue of how to provide people with these tools in three ways. it provides a theoretical framework for the design of small pieces of software, Agents, which communicate using usefully-structured information, Workslips, and can combine to form helpful systems for people, Personal Agent Systems. It provides a set of philosophical guide-lines which suggest particular design choices developers should make while developing these Agents. Finally it provides a reference implementation, TWPARI-J, which can be used to develop these Agents and provides practical examples of the philosophical guidelines. To demonstrate the effectiveness of TWPARI-J and the applicability of the philosophical guidelines and conceptual framework, three example systems are presented which allow people to: adjust their alarm clocks according to the weather, plan their shopping according to sales prices of local stores, and to get assistance monitoring the state of their network in an intelligent fashion. For each system particular instantiations are provided where the system is able to help a particular person. The scheme provides a simple system to create personal assistant system, often without needing to write new software, and simplifies Agent development. The scheme allows people to use the system and develop applications with a maximum of flexibility using a minimum of work.
  • Thumbnail Image
    ItemOpen Access
    Visualization of Multivariate Data on Surfaces
    (2019-03-19) Rocha, Allan; Costa Sousa, Mario; Alim, Usman Raza; Chan, Sonny; Jacob, Christian J.; Geiger, Sebastian; Tominski, Christian
    In several domains of science and applications, the understanding of scientific data leads to technological advances and scientific discovery. Multivariate 3D data, for example, is essential for decision-making in fields such as Medicine and Geology, where experts are required to understand and correlate several spatial attributes. To simplify complexity and facilitate understanding, the 3D data is often explored through surfaces of interest. This is the reason why the visualization of multivariate data on surfaces has been a topic of interest among the visualization community. However, much work has been needed to provide visualization solutions that facilitate the multivariate visualization design, creation, and exploration. This research builds upon ideas introduced and discussed many years ago that focus on the problem of visualizing multiple attributes on surfaces in a single view. Here I present a new perspective to this problem as well as a solution that allows us to design, visualize and interact with multivariate data on surfaces. This perspective is created from the combination of several aspects born in fields such as Illustration, Perception, and Design, that have been employed and studied by the visualization community both in Information and Scientific Visualization. Therefore, this thesis lies between these two main fields, since it involves aspects from both. By building upon this multidisciplinary combination, I present a new way to visualize multivariate data on surfaces by exploiting the concept of layering. First, I introduce a new real-time rendering technique and the concept of Decal-Maps, which fills a gap in the literature and allow us to create 2D visual representations such as glyphs that follow the surface geometry. Building on this technique, I propose the layering framework to facilitate the multivariate visualization design on surfaces. The use of this concept and framework allows us to connect and generalize concepts established in flat space, such as 2D maps, to arbitrary surfaces. This thesis also demonstrates that the design of new multivariate visualizations on surfaces opens up other new possibilities such as the use of interaction techniques. Here I demonstrate this potential by introducing a new interaction technique that allows us to explore multivariate data and to create customized focus+context visualizations on surfaces. This is achieved by introducing a new category of lenses, Decal-Lenses, which extends the concept of magic-lenses from flat space to general surfaces. Finally, this thesis showcases the process of multivariate visual design and data exploration through a series of examples from several domains. Inspired by these examples, I also contribute with an in-depth application research conducted from my long-term collaboration with domain experts in the fields of Geology and Reservoir Engineering. This application illustrates how the proposed approach can support and facilitate decision-making in the complex process of Geological Modelling.