BAB III KOMPONEN MULTIMEDIA
1. Multimedia Building Blocks •
Komponen aplikasi multimedia terdiri dari : a. Graphics -
Background
- Organizational chart
-
Photographs
- Line Art
-
Grafik 3 dimensi
- Clip Art
-
Chart
- Buttons
-
Flowchart
-
Tittles
-
Buttons
-
Bullets
-
Paragraphs
-
Scrolling Text
b. Text
c. Video -
Digitized video
-
Video window Æ menampilkan output dari video player
-
Video window Æ menampilkan siaran TV
d. Sound -
Sound effect
-
Naration
-
Audio tracks
-
Background music
-
Background / ambient sounds
e. Animasi -
Path animation (mengubah posisi, gerak)
-
Animasi 2,5 D (bayangan, higklight, perspektif)
-
Animasi 3D Page 1 of 30
A. Graphics •
Komponen grafis merupakan unsur yang mendominasi sebuah presentasi multimedia Æ “a picture is worth a thousand words”
•
Kegunaan grafis dalam aplikasi multimedia : a. Sebagai ilustrasi untuk menjelaskan kosep-konsep b. Chart Æ untuk ilustrasi dan meringkas data-data numerik c. Warna, bakground dan icon untuk menyediakan keseragaman dan keberlanjutan dalam aplikasi d. Integrasi dari text, photo, dan grafik untuk mengekspresikan konsep, informasi atau suasana hati e. Menunjukkan image dan budaya perusahaan f. Simulasi dari lingkungan yang ada g. Menjelaskan proses h. Menjelaskan struktur organisasi i. Ilustrasi dari lokasi
•
Jenis-jenis graphics : a. Backgrounds b. Photos c. Grafik 3-dimensi d. Charts (graphs) e. Flow chart f. Organizational charts g. Buttons
Page 2 of 30
•
Alur manipulasi graphics dalam pengembangan aplikasi multimedia
Content Specialist
Graphic Artist
Photos and Images
Scan
Multimedia Authoring Specialist
Image editing
Authoring
Logos, graphics, etc Graphic development Colors, backgrounds, textures
Clipart and photo CD collection
1. Backgrounds •
Merupakan tema dari aplikasi yang dibuat
•
Juga menunjukkan kompleksitas produksi khususnya dalam penggunaan grafik 3-dimensi
•
Pemilihan disain background bergantung pada : a. Tema aplikasi b. Kapasitas resolusi warna proyektor atau monito yang digunakan untuk menampilkan aplikasi c. Kapasitas media penyimpan yang digunakan dalam distribusi d. Jumlah text yang akan ditempatkan di depan background, contoh : -
Jika banyak text yang harus ditampilkan Æ background harus sederhana agar text dapat terbaca dengan jelas
Page 3 of 30
-
Jika aplikasi akan ditampilkan di internet, maka semakin banyak graphics akan memperlambat waktu download
2. Parameter dalam pemilihan gambar digital : a. Bit / color resolution -
Merupakan ukuran jumlah bit yang disimpan per pixel yang menunjukkan jumlah warna yang dapat ditampilkan dalam waktu bersamaan di layar
-
Ukuran yang ada : 1. 1 bit (1 color, 1 black) 2. 2 bit (4 colors) 3. 4 bit (16 colors) 4. 8 bit (256 colors) 5. 16 bit (65.536 colors) Æ hi-color 6. 24 bit (16.7 million colors) Æ true color
b. Device / output resolution -
Menunjukkan jumlah dots per inch (dpi) yang dapat dihasilkan oleh piranti keluaran seperti monitor, LCD panel, atau video proyektor
-
Contoh : 300-dpi (dots per inch) printer is one that is capable of printing 300 distinct dots in a line 1 inch long. This means it can print 90,000 dots per square inch.
-
Resolusi monitor biasanya 72 dpi
c. Screen resolution -
Menunjukkan jumlah dpi dalam halftone yang digunakan oleh layar untuk menampilkan gambar abu-abu atau separasi warna
-
Resolusi layar diukur dalam satuan lines per inch (lpi)
d. Image resolution -
Menunjukkan jumlah informasi yang tersimpan untuk sebuah warna; diukur dalam satuan pixels per inc (ppi)
Page 4 of 30
-
Semakin tinggi resolusi gambar Æ semakin besar ukuran file Æ semakin lama di-load Æ semakin lama di-print
•
Contoh ukuran : 640 x 480, 800 x 600, 1024 x 768
TIPS : a. Pada saat men-scan gambar / photo pilih resolusi antara 300 dpi s.d 600 dpi b. Ukuran gambar tidak boleh melebihi resolusi layar, pilihlah ukuran standar 640 x 480 pixel c. Pada saat meng-export gambar ke aplikasi multimedia turunkan resolusi menjadi 72 dpi (menyesuaikan dengan resolusi layar)
3. Format file •
Pada umumnya format file ditentukan oleh pengembang software. Misalnya Apple macintosh menggunakan format PICT sebagai format standar
•
Pengelompokan format gambar : a. Bitmaps -
Adalah gambar yang disimpan sebagai sekumpulan pixel yang bersesuaian dengan grid dari titik-titik pada layar monitor
Format name
Windows Bitmap
File
Type of
Extension
image file
.bmp
Bitmap
Intended puspose
Format paling efisien dalam windows
Drawing Exchange File
.dxp
Vector
Encapsulated Post Script
.eps, .epsf
Vector
GIF
.gif
Bitmap
Graphics
Interchange
Format; banyak dipakai di internet GEM File
.img
Page 5 of 30
Bitmap
Initial Graphic Exchange
.img, .iges
Vector
JPEG
.jpg, .jpeg
Bitmap
Joint Photographic Experts Group; merupakan platform yang independen
MPEG
.mpg, .mpeg
Bitmap
Auto CAD
.plt
Vector
Photo CD
.pcd
Bitmap
Dikeluarkan Kodak
PICT / PICT2
.pct, .pict
Bitmap
Standar format Macintosh
Targa
.tga
Bitmap
True Vision Targa format; hasil capture dari video
MS Word Meta File
.wmf
Bitmap
Word Perfect Graphics
.wpg
Bitmap
Format
graphics
dari
WordPerfect TIFF
.tif
Bitmap
Tagged Image File Format (TIFF); terlalu banyak subformat
Device-independent bitmap
.dib
Bitmap
Digunakan untuk transfer bitmap dari satu device ke device lain
Paintbrush graphics format
.pcx
Bitmap
Dikeluarkan Paintbrush;
Zsoft untuk
under
DOS PC paint graphics format
.pic
Bitmap
Dikeluarkan oleh PC Paint
Portable Network Graphics
.png
Bitmap
Portable Network Graphics, telah
dipatenkan
mengganti
format
untuk GIF;
disetujui World Wide Web Concortium (W3C)
b. Vector image
Page 6 of 30
-
Adalah
gambar
yang
terimpan
sebagai
persamaan
matematik (disebut algoritma) yang menunjukkan kurva, garis, dan bentuk-bentuk lain -
Mudah untuk diperbesar / diperkecil tanpa menurunkan kualita gambar; ukuran file lebih kecil dibandingkan dengan bitmaps
B. Text •
Pada awal sejarah peradaban, manusia telah menggunakan gambargambar
dan
tulisan
untuk
menceritakan
tentang
pengalaman,
pengetahuan, dan perasaan mereka •
Teks merupakan alat komunikasi yang utama, jauh sebelum Gutenberg menemukan mesin cetak
•
Dengan perkembangan teknologi Multimedia, teks dapat dikombinasikan dengan media lain dengan cara yang lebih powerful dan bermakna untuk menyajikan informasi dan mengekspresikan perasaan
•
Teks dapat dirancang dengan menggunakan : a. Word Processor (WP) -
Teks dibuat menggunakan WP kemudian di import dari Multimedia
Authoring
Program
seperti
Macromedia
Director atau Macromedia Authorware dalam format Rich Text Format (RTF) b. Authoring Software (AS) -
Teks dibuat menggunakan fasilitas text editor yang terdapat dalam program seperti Macromedia Director
•
Hal-hal yang harus diperhatikan dalam menggunakan teks dalam aplikasi multimedia : a. Pahami kegunaan aplikasi yang dibuat b. Jumlah teks yang digunakan c. Jenis / type font yang dipakai
Page 7 of 30
d. Ukuran dan warna font •
Bedakan penggunaan teks untuk : a. Individual user -
Untuk aplikasi multimedia yang digunakan oleh seorang pemakai dengan menggunakan komputer Æ gunakan extensive
text
dengan
ukuran
yang
lebih
kecil
dibandingkan untuk group presentation b. Group presentations -
Gunakan teks seminimal mungkin; karena yang berbicara adalah presenter / penyaji. Teks hanya sebagai panduan saja
-
Batasi penggunaan teks hanya pada bullet text atau paragraf pendek
-
Ukuran font minimal 24 points
1. Format Text •
Format teks dapat dibagi menjadi 2 bentuk : a. Bullet text -
Berisi teks pendek; diawal dengan simbol-simbol tertentu
-
Digunakan untuk menjelaskan konsep atau menjelaskan suatu maksud
-
Contoh simbol yang digunakan : y, , 1, a, dll
b. Paragraf text -
Merupakan sekumpulan teks; biasanya terdiri lebih dari 1 kalimat
-
Ada 4 bentuk format paragraf, yaitu : left-aligned, rightaligned, centered, justified
•
Untuk paragraf yang panjang Æ gunakan scrolling text
•
Apabila teks di export / ditampilkan sebagai elemen grafis Æ maka text dapat dimanipulasi / ditambahkan efek-efek khusus seperti : blending, strecthing, resizing, changing color, adding fills, and shadows
Page 8 of 30
2. Font type •
Pada umumnya ada 2 type font yang dikenal komputer : a. Bitmap fonts -
Tersimpan sebagai karakter tunggal, yang terdiri dari kumpulan titik-titik yang nampak di layar ketika diperlukan
-
Apabila ukuran font tidak terinstall di komputer maka font akan disesuaikan ukurannya secara matematis sehingga hasilnya kurang bagus
b. Outline fonts (truetype fonts dan multiple master) -
Terdiri dari outlines yang terisi suatu warna / objek yang terbentuk setiap kali digunakan
-
Selalu kelihatan halus dan bentuknya rata berapapun ukurannya
•
Agar font yang digunakan dapat tampil di setiap platform (cross-platform application) maka font harus di convert ke bitmap font
3. Point sizes (pts) •
Point adalah pengukuran tipografi yang kurang lebih setara dengan 1/72 inci
•
Contoh : 4 point Æ 1/72 x 4 = 0,05 inci
4. Format karakter •
Ada beberapa format karakter, yaitu : a. Bold Æ Bold b. Caps / Small caps Æ Caps / SMALL CAPS c. Underline Æ Underline d. Outline Æ e. Condensed Æ Condensed f. Superscript Æ Superscript g. Shadow Æ Shadow h. Emboss Æ EEm mbboossss
Page 9 of 30
i. Engrave Æ EEnnggrraavvee j. All caps Æ ALL CAPS k. Italic Æ Italic l.
Subscript Æ Subscript
m. Strikethrough Æ Strikethrough n. Character spacing Æ C h a r a c t e r s p a c i n g o. Color Æ Color p. Double underline Æ Double underline •
Untuk hal-hal tertentu, gunakan gabungan dari beberapa format karakter
•
Untuk penulisan judul presentasi gunakan spasi antar karakter yang berbeda dengan kalimat lain. Defaultnya adalah 1 pts
4. Font Moods •
Font moods adalah perasaan yang terkait dengan font style yang dipilih, yang dipengaruhi oleh kondisi sosial dan teknologi yang berpengaruh pada disain
•
Digunakan untuk memperjelas pesan yang digunakan
•
Perasaan (moods) dipengaruhi oleh : a. Personal taste (citra diri) b. Experience (pengalaman) c. Educations (pendidikan) d. Cultural backgrounds (latar belakang budaya)
•
Font moods yang sering dipakai : a. Trendy -
Representasi dari hal-hal yang populer pada saat itu
-
Font : impact
b. Nostalgic -
Melambangkan masa-masa kuno, awal abad 20
-
Font : bodoni
c. Traditional -
Menunjukkan perasaan pemakai yang menyenangkan
-
Font : Bookman
Page 10 of 30
d. Classic -
Menunjukkan keseimbangan yang alami / klasik
-
Font : New York
e. Playfull -
Menghibur dan santai
-
Font : Mistral
f. Agresive -
Mengharap perhatian dan tanggapan dari pemakai atau audience
-
Font : Helvetica
g. Friendly -
Nyaman, mudah dibaca, pendekatan secara personal
-
Font : Arial
h. Informative -
Menunjukkan bahwa penyampaian informasi merupakan tugas yang paling penting
-
Font : Times Roman
i. Sophisticated -
Sesuatu yang cantik dan atraktif
-
Font : Wide Latin
C. Video •
Suara dan video memegang peranan yang sangat penting dalam presentasi multimedia
•
Sound merupakan dimensi aural yang menentukan mood dan tercapainya tujuan presentasi
•
Video telah diperkenalkan kurang lebih 50 tahun yang lalu. Namun hubungan antara video (televisi) dan komputer merupakan hal yang relatif masih baru; sedangkan digital video merupakan teknologi yang lebih baru lagi.
•
Orang akan lebih tertarik dengan aplikasi / presentasi yang menampilkan tayang dalam bentuk video
Page 11 of 30
•
Proses digital video : a. Analog video acquisition -
Pembuatan script, Penyusunan storyboard, pemilihan peraltan untuk pengambilan gambar
b. Capturing and Storing c. Editing and Adding Special effects d. Delivery and Display Format Video •
Video dibedakan dalam dua format, yaitu : a. Analog : NTSC dan PAL b. Digital : MOV, MPG, AVI, ASF, dll
•
Proses mengubah dari analog ke format digital disebut dengan Capturing atau sampling. Proses capturing memerlukan alat yang berupa video capture board atau frame grabber yang dipasang dalam komputer, yang berfungsi untuk merubah sinyal analog menjadi sinyal digital.
•
Semakin lama durasi video analog Æ semakin beasr RAM dan harddisk yang dibutuhkan untuk menyimpannya dalam format digital. Sebagai ilustrasi : 1 frame digital video dengan kualitas 24 bits membutuhkan sekitar 1 Mb. 10 detik digital video – full screen – full motion memerlukan 300 Mb
•
Video digital mengalami kompresi antara 1/50 hingga 1/200 dari ukuran aslinya JPEG
Joint Photographic Experts Group Rasio kompresi 20 : 1 Merupakan algoritma kompresi yang paling banyak dipakai
MPEG
Moving Picture Experts Group Rasio kompresi 50 : 1; mampu melakukan kompresi hingga 200 : 1 namun mengalami penurunan kualitas gambar
DVI •
Rasio kompresi 80 : 1 hingga 160 : 1
Hal-hal yang perlu diperhatikan dalam mengolah video : o Data transfer rate dari media penyimpan
Page 12 of 30
o Ukuran jendela
Merupakan
jumlah
pixel
yang
ditampilkan
secara
horisontal dan vertikal. Ukuran jendela mempengaruhi kualitas video dan kapasitas media penyimpan. Ukuran standard : 160x120, 240x180, 320x240 o Frame rate
Menunjukkan jumlah frame per seconds (fps) selama video dimainkan. Semakin endah frame rate maka semakin cepat data dapat diproses dan semakin kecil ukuran file video. Format standard 24 fps dan 30 fps
o Kualitas gambar dan resolusi
Merupakan kedalaman bit dari format digital yang akan dicapture oleh encoder (codec). Nilai standarnya 8 hingga 24 bits. Semakin rendah angkanya maka jumlah warna yang bisa ditampilkan semakin sedikit, tapi proses transfer semakin cepat.
Ukuran yang normal : 240x180, 15 hingga 24 fps, CDROM sbg distribusi
o Kapasitas media penyimpan Video Compression Codecs (Compression decompressions device) No.
Codec
1.
Apple
Kegunaan Animation, Hemat media penyimpan, tidak efisien
Apple None 2.
Cinepak
Kualitas CDROM, paling banyak dipakai
3.
H.261
Low-quality video conferencing
4.
H.263
Medium-quality video conferencing
5.
Intel Indeo 3
Medium-quality CDROM video
6.
Intel
Indeo High-quality CDROM video
Interactive 7.
Motion JPEG
General purpose
8.
Photo JPEG
Menghasilkan file yang kecil
Page 13 of 30
9.
MPEG-1
High-quality CDROM video, butuh alat khusus
10.
MPEG-2
High-quality DVDROM video, butuh alat khusus
11.
MPEG-4
High-quality web-based video
12.
Sorenson
High-quality video untuk publikasi di internet & CDROM
D. Animasi A simulation of movement created by displaying a series of pictures, or frames. Cartoons on television is one example of animation. What is Animation? • 50 years ago Walt Disney created animated objects such as Mickey Mouse. • Today the process used to create animated objects has had to change. • In fact, it continues to change. Definition • The word “animation” is a form of “animate,” which means to bring to life. • Animation is the use of computer to create movement on the screen • Thus when a multimedia developer wants to bring an image to life, animation is used. • For example, a spinning globe is it better to film the motion on video, or is animation a better solution.
There are four kinds of animation 1. Frame Animation o Makes object by displaying a series of predrawn pictures, called frames, in which the objects appear in different locations on the screen o In a movie, a series of frames moves through the film projector at about 24 fps. Why 24 fps ? because that is the threshold beneath which you would notice flicker or jerkiness on the screen 2. Vector Animation o Vector is a line that has a beginning, a direction, an a length.
Page 14 of 30
o Vector animation makes objects move by varying these three parameters for the line segments that define the object o Software : macromedia flash 3. Computational Animation o You move objects across the screen simply by varying their x (horizontal position) and y (vertical position) coordinates. 4. Morphing o Morphing means to transition one shape into another by displaying a series of frames that creates a smooth movement as the first shape transform itself into the other shape o Software: Avid’s Elastic Reality, Black Belt’s WinImages, Gryphon Software’s Morph, Human Software’s Squizz, MorphWizard, Unlead’s MorphStudio
Models 1. Cel models • early animators drew on transparent celluloid sheets or cels, different sheets contained different parts of the scene, which was assembled by overlaying the sheets • in animation, cels are digital images with a transparency channel • scenes are rendered by drawing the cels back to front, with movement being added by changing the position of cels from one frame to the next • a cel model is therefore a set of images, their back to front order, and their relative position and orientation in each frame 2. Scene-based models • simply a sequence of graphics models, each representing a complete scene • highly redundant and do not support continuity of activities 3. Event-based models • expresses the difference between successive scenes as events that transform one scene to the next
Page 15 of 30
• still discrete rather than continuous, but permits the management of scenes by input devices (i.e. mouse, tablet, etc.) rather than each scene having to be entered manually 4. Key frames • in essence, the animator models the beginning and end frames of a sequence and lets the computer calculate the others by interpolation 5. Articulated objects & hierarchical models • attempt to overcome the problems of key frames by developing articulated objects, jointed assemblies where the configuration and movement of subparts are constrained • ensures proper relative positioning and constraint maintenance during interpolation (will not allow solid objects to pass through other solid objects) 6. Scripting and procedural models • current state-of-the-art animation modelling systems have tools allowing the animator to specify key frames, preview sequences in real time and control the interpolation of model parameters • an additional feature in many such systems are scripting languages • scripting languages offer the animator the opportunity to express sequences in concise form, particularly useful for repetitive and structured motion and also provide high-level operations intended specifically for animation 7. Physically-based models & empirical models • this approach is used to produce sequences depicting evolving physical systems • a mathematical model of the system is derived from physical principles or empirical data and the model is then solved, numerically or through simulation, at a sequence of time points, each one resulting in a single frame for the sequence Animation Operation • Graphics operations
Page 16 of 30
since animation models are graphics models extended in time, all the graphics operations we have already covered are applicable here
• Motion and parameter control
since the essential difference between graphics and animation operations is the addition of the temporal dimension, graphics objects become animations through the assignment of complex trajectories or behaviours over time
commercial 3D animation systems provide modelling tools and animation tools, the modelling tools produce 3D graphic models and the animation tools add temporal transformations to these objects
Principles of Animation • Animation is possible because of a biological phenomenon known as persistence of vision And The psychological phenomenon called phi . • An object seen by the human eye remains chemically mapped on the eye’s retina for a brief time after viewing. • Combined with the human mind’s need to conceptually complete a perceived action. • This makes it possible for a series of images that are changed very slightly and very rapidly, one after the other, seem like continuous motion . Approaches to animation • Straight ahead Draw/animate one frame at a time Can lead to spontaneity, but is hard to get exactly what you want • Pose-to-pose Top-down process: Plan shots using storyboards -- Plan key poses first -- Finally fill in the in-between frames
Page 17 of 30
Keyframe animation • Keyframing is the technique used for pose-to-pose animation o Head animator draws key poses—just enough to indicate what the motion is supposed to be o Assistants do “in-betweening” and draws the rest of the frames o In computer animation substitute “user” and “animation software” o Interpolation is the principal operation
Kinematics • Kinematics is the study of the movement and motion of structures that have joints, such as a walking man. o Forward o Inverse • Software: Fractal Design’s Poser • Sample : robotics
Page 18 of 30
Animation Principles • Timing • Ease in/Out • Moving in arcs • Anticipation • Follow through • Staging • Weight Animation Techniques • Computers have taken a great deal of handwork out of the animation and rendering process. • And commercial films such as Jurassic Park, Beauty and the Beast, Toy Story, and Shrek have utilized the power of the computers. Motion Capture • A method for creating complex motion quickly: measure it from the real world
Page 19 of 30
• Types :
Page 20 of 30
Motion capture data processing • Marker identification: which marker is which o Start with standard rest pose o Track forward through time (but watch for markers dropping out due to occlusion!) • Calibration: match skeleton, find offsets to markers o Use a short sequence that exercises all DOFs (degrees of freedom) of the subject o A nonlinear minimization problem • Computing joint angles: explain data using skeleton DOFs o A inverse kinematics problem per frame!
Basic surface deformation methods • Mesh skinning: deform a mesh based on an underlying skeleton • Blend shapes: make a mesh by combining several meshes
Both use simple linear algebra • Easy to implement—first thing to try • Fast to run—used in games
Page 21 of 30
Page 22 of 30
2-D Animation 1. Cel animation • Made famous by Disney • 24 frames per second therefore a minute may require as many as 1,440 separate frames. • Cel animation: is based on changes that occur from one frame to the next. • Cel stands for celluloid which is a clear sheet with images drawn on them. • The celluloid images are place on a background that is usually stationary. • The background remain fixed as the images changes.
2. Path animation • Moves an object along a predetermined path on the screen • The path can be a straight line or have a number of curves. • Starts with keyframes (the first and last frame of an action). • The series of frames in between the keyframes are drawn in a process called tweening. • Tweening requires calculating the number of frames between keyframes and the path the action takes, and then actually takes, and then sketches a series of progressively different outlines.
Computer Animation • Typically employees the same logic and procedural concepts as cel animation • You can usually set your own frame rate • At 15 frames a second the animation may appear jerky and slow • 2-D animation can be an acceptable alternative to the expense of creating video 3-D Animation • 3-D Animation involves three steps: modeling, animation, and rendering o Modeling – the process of creating objects and scenes
Page 23 of 30
o Animation – the process of defining the object’s motion o Rendering – the final step in creating 3-D animation. 2 basic forms:
real-time - model is rendered as frames
are displayed, 10+
frames per second are required to avoid jerkiness, so only appropriate for simple models or with special hardware
non-real-time -frames are pre-rendered, taking as long as necessary to do so, provides higher visual quality and consistency of frame-rate
Page 24 of 30
Animation File Formats Software Director Animator Pro Studio Max SuperCard and Director Windows Audio Video Interleaved Macintosh Motion Video CompuServe Flash Shockwave
File Format .dir & .dcr .fli .max .pics .avi .qt & .mov .mpeg .gif .swf .dcr
E. Digital audio • Digital Audio Representation 2 main areas : o telecommunications o entertainment (audio CD) • Produced by sampling a continuous signal generated by a sound source. An analog-to-digital converter (ADC) takes as input an electrical signal corresponding to the sound and converts it into a digital data stream. The reverse process, to generate the sound through an amplifier and speakers, involves a digital-to-analog converter (DAC)
Sampling • Sampling frequency (rate) o sampling theory shows that a signal can be reproduced without error from a set of samples, providing the sampling frequency is at least twice the highest frequency present in the original signal o telephone networks allocate a 3.4kHz bandwidth to voice-grade lines, thus a sampling rate of 8kHz is used for digital telecommunications o the human ear is sensitive to frequencies of up to about 20kHz, so to digitise any perceivable sound a sampling rate of over 40kHz is required • Sample size and quantisation
Page 25 of 30
o during sampling, the continuously varying amplitude of the analog signal is approximated by digital values, this introduces a quantisation error, being the difference between the actual amplitude and the digital approximation o quantisation error is apparent when the signal is reconverted to analog form as distortion, a loss in audio quality o quantisation error can be reduced by increasing the sample size, as allowing more bits per sample will improve the accuracy of the approximation Quantisation • Quantisation refers to breaking the continuous range of the analog signal into a number of unique digital intervals, based on one of a number of schemes: o linear quantisation - uses equally spaced intervals, so if the sample size is 3 bits and the maximum signal variation is 5.0 then the quantisation interval would be 0.625 units of signal amplitude o nonlinear quantisation (especially logarithmic quantisation) - uses nonequally spaced intervals, lower amplitude intervals are more closely spaced than higher amplitude, results in greater sensitivity to lower amplitude sound where the human ear is most sensitive • Number of channels (tracks) o speech quality audio is mono (1 track) o stereo audio requires 2 tracks o some consumer audio equipment use 4 tracks (quadrophonic) o professional audio equipment uses 16, 32 or more
Digital audio format • Interleaving
Page 26 of 30
o a multi-channel audio value can be encoded by interleaving channel samples or by providing separate streams for each channel o the advantage of interleaving is in synchronisation, and it also offers some benefits in storage and transmission o the disadvantages of interleaving are that it can be wasteful of space or bandwidth if not all channels are needed, it freezes the synchronisation between channels thus preventing temporal shifts, and it may not allow variation in the number of channels • Negative samples o the voltages found in analog audio signals alternate between positive and negative values o negative values can be encoded successfully for processing in twos complement, ones complement or sign-magnitude representation
Encoding • Encoding audio data reduces storage and transmission costs, and compressed audio also provides better quality when compared to uncompressed audio at the same data rate • 2 commonly-used methods: o PCM (Pulse Code Modulation) - uses the fact that a digital signal can be formed from a series of pulses (see diagram). PCM values are simply sequences of uncompressed samples, so they provide a reference format for comparison with more complex coding methods o ADPCM (Adaptive Delta Pulse Code Modulation) - reduces PCM data rate by encoding the differences between samples. ADPCM is widely used and is associated with some encoding standards, such as CCITT G.721 Digital Audio Operations • Storage o it is possible to record digital audio, even at the data rates of the high quality formats, on general purpose magnetic storage
Page 27 of 30
o theoretically, a magnetic disk with a sustainable transfer rate of 5 Mbytes per second could playback 50 channels of CD-quality digital audio. In practice this would not be possible without a highly optimised layout, but one or two channels are easily within the reach of small computer systems o since an hour of stereo digital audio, at the CD data rate, requires over half a Gigabyte of storage, tertiary storage in the form of DAT tapes, CD discs or optical disks is normally adopted, with the information being mounted onto the system manually or through a jukebox
Retrieval • Need to support random access and ensure continuous flow of data to DAC • portions of audio sequences, segments, are identified by their starting time and duration, these can be located is by mapping the starting time to a segment address, which the file system then maps to a physical address on disk o where there is no direct mapping to enable segment location by time code, an index of segments must be separately maintained o continuous flow of data is easy to maintain with a dedicated storage system, but requires careful control where storage is scheduled for a number of such tasks
Editing • as with digital video, 2 types : o tape-based o disk-based • to avoid audible clicks when inserting one sample into another, cross-fades are used, where the amplitudes of the original segment and the inserted segment are added and scaled about the insertion point • digital audio also supports non-destructive editing, where the segments of data are accessed through a data structure known as a play-list, which
Page 28 of 30
essentially contains a set of pointers to the data and details on ordering and other forms of edit to be performed on the data when it is joined
Effects • Effects and filtering o digital filtering techniques permit a number of effects on audio : -
Delay
-
Equalisation & Normalisation
-
Noise reduction & Time compression and expansion
-
Pitch shifting
-
Stereoisation
-
Acoustic environments
• Conversion o one format to another (uncompressing ADPCM Æ PCM) o altering encoding parameters (i.e. resampling at lower frequency)
MIDI • the Musical Instrument Digital Interface was developed in the early ‘80s by musical equipment makers Devices : o electronic keyboards and synthesisers o drum machines o sequencers (to record and play back MIDI messages) o music<->film and music<->video synchronisation equipment • MIDI Concepts: o Channel - a MIDI connection has 16 message channels, devices can be set to respond to all channels or only to specific channels o Key number - notes are identified by key number, 128 compared with a standard keyboard of 88
Page 29 of 30
o Controller - 128 different controllers are available under the MIDI protocol, though not all are currently defined, changing the value of a controller typically alters sound production o Patch/program - an audio palette is called a program or patch, a synthesiser capable of having a number of patches active at the same time is called multi-timbral o Polyphony - the ability of a synthesiser to play many notes at a time o Song - a recorded or preprogrammed MIDI sequence o Timing clock - a MIDI sequencer timestamps messages using a timebase measured in parts per quarter note (PPQ). Typical timebase values are 24, 96 and 480 PPQ. To convert the timebase into actual time you use the tempo, measured in beats per minute (BPM) where we assume that one beat is equal to a quarter note. Thus if we have a tempo of 180 BPM, a time base of 96PPQ = 1/3 x 1/96 = 3.47ms o MIDI synchronisation - MIDI devices can be set to internal synch or external synch, when set to internal synch a device is known as a master and produces a timing clock message on its MIDI OUT at 24PPQ which slave devices use for external synch o MTC - MIDI Time Code is used to synchronise MIDI with film or video, used to trigger sound effects or musical sequences • Limitations of MIDI : o operates at 31250bps, allows 500 notes per second which may not be enough for complex pieces o limited number of channels, lack of device addressing and other flaws make configuring large MIDI networks difficult o device dependence of MIDI data
Page 30 of 30
