{"id":49,"date":"2025-01-30T17:52:08","date_gmt":"2025-01-30T17:52:08","guid":{"rendered":"https:\/\/citelearn3.savecicadabuzz.org\/?page_id=49"},"modified":"2025-02-27T17:36:07","modified_gmt":"2025-02-27T17:36:07","slug":"week-2-call","status":"publish","type":"page","link":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/week-2-call\/","title":{"rendered":"Week 2- CALL"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-page\" data-elementor-id=\"49\" class=\"elementor elementor-49\">\n\t\t\t\t<div class=\"elementor-element elementor-element-9a2f75e e-flex e-con-boxed e-con e-parent\" data-id=\"9a2f75e\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-aee9df2 e-con-full e-flex e-con e-child\" data-id=\"aee9df2\" data-element_type=\"container\">\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-1d55bc5 e-con-full e-flex e-con e-child\" data-id=\"1d55bc5\" data-element_type=\"container\">\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-96282af e-flex e-con-boxed e-con e-parent\" data-id=\"96282af\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ea04c51 elementor-widget elementor-widget-text-editor\" data-id=\"ea04c51\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-014b082 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"014b082\" data-element_type=\"section\"><div class=\"elementor-container elementor-column-gap-default\"><div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-456759a\" data-id=\"456759a\" data-element_type=\"column\"><div class=\"elementor-widget-wrap elementor-element-populated\"><div class=\"elementor-element elementor-element-b51009f elementor-widget elementor-widget-heading\" data-id=\"b51009f\" data-element_type=\"widget\" data-widget_type=\"heading.default\"><div class=\"elementor-widget-container\"><p dir=\"ltr\">Is white space tokenization enough?<br \/>In this assignment, you will use an online tokenization tool. Navigate to <a href=\"http:\/\/text-processing.com\/demo\/tokenize\/\">http:\/\/text-processing.com\/demo\/tokenize\/<\/a>\u00a0 and try to following:\u00a0<\/p><ol><li dir=\"ltr\" aria-level=\"1\"><p dir=\"ltr\" role=\"presentation\">Enter several sample sentences (you can copy paste them from the web or write your own) into the textbox where it says \u201ctokenize text\u201d. Your sentences should include at least one contraction and at least one compound word (if you don\u2019t know what a compound word is, see<a href=\"https:\/\/www.grammarly.com\/blog\/open-and-closed-compound-words\/\"> here<\/a>).<\/p><\/li><\/ol><p><span style=\"font-weight: 400;\">Sentences used:<\/span><\/p><ul><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">She <\/span><b>doesn\u2019t<\/b><span style=\"font-weight: 400;\"> want to miss the <\/span><b>fireworks<\/b><span style=\"font-weight: 400;\"> at the festival.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">You <\/span><b>could\u2019ve<\/b><span style=\"font-weight: 400;\"> warned me that the <\/span><b>football<\/b><span style=\"font-weight: 400;\"> game was canceled.<\/span><\/li><li><span style=\"font-weight: 400;\">He <\/span><b>should\u2019ve<\/b><span style=\"font-weight: 400;\"> known that the <\/span><b>shortcut<\/b><span style=\"font-weight: 400;\"> would actually take longer.<\/span><\/li><\/ul><ol><li dir=\"ltr\" aria-level=\"1\"><p dir=\"ltr\" role=\"presentation\">Observe how the different tokenizers handle your text. Look carefully at the whitespace tokenizer and answer the following question: Are spaces sufficient to tokenize English language text? Why or why not? Cite examples from your test to support your conclusion.<\/p><\/li><\/ol><\/div><\/div><\/div><\/div><\/div><\/section><section class=\"elementor-section elementor-top-section elementor-element elementor-element-30c8446 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"30c8446\" data-element_type=\"section\"><div class=\"elementor-container elementor-column-gap-default\"><div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-5427147\" data-id=\"5427147\" data-element_type=\"column\"><div class=\"elementor-widget-wrap elementor-element-populated\"><div class=\"elementor-element elementor-element-ae49b69 elementor-widget elementor-widget-heading\" data-id=\"ae49b69\" data-element_type=\"widget\" data-widget_type=\"heading.default\"><div class=\"elementor-widget-container\"><p><span style=\"font-size: 16px; font-weight: 400;\">No, spaces are not sufficient to tokenize English language text due to various factors like the punctuation in contractions, hyphenated words, and compound words, that introduce different levels of complexity to language. These examples must be separated deeper than spaces, as they have different principle parts that set them apart from simple, one or two syllable words.\u00a0<\/span><\/p><article><div data-elementor-type=\"wp-page\" data-elementor-id=\"138\" data-elementor-title=\"Page\"><section data-id=\"014b082\" data-element_type=\"section\" data-model-cid=\"c36\"><div data-id=\"456759a\" data-element_type=\"column\" data-model-cid=\"c37\" data-col=\"100\"><div data-id=\"b51009f\" data-element_type=\"widget\" data-model-cid=\"c38\" data-widget_type=\"heading.default\"><p><br \/><br \/><\/p><div>\u00a0<\/div><p>\u00a0<\/p><\/div><\/div><\/section><\/div><\/article><\/div><\/div><\/div><\/div><\/div><\/section><section class=\"elementor-section elementor-top-section elementor-element elementor-element-fff16ef elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"fff16ef\" data-element_type=\"section\"><div class=\"elementor-container elementor-column-gap-default\"><div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f0d008c\" data-id=\"f0d008c\" data-element_type=\"column\">\u00a0<\/div><\/div><\/section>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Is white space tokenization enough?In this assignment, you will use an online tokenization tool. Navigate to http:\/\/text-processing.com\/demo\/tokenize\/\u00a0 and try to [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"site-sidebar-layout":"no-sidebar","site-content-layout":"","ast-site-content-layout":"full-width-container","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"disabled","ast-breadcrumbs-content":"","ast-featured-img":"disabled","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"class_list":["post-49","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/pages\/49","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/comments?post=49"}],"version-history":[{"count":9,"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/pages\/49\/revisions"}],"predecessor-version":[{"id":177,"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/pages\/49\/revisions\/177"}],"wp:attachment":[{"href":"https:\/\/citelearn3.savecicadabuzz.org\/index.php\/wp-json\/wp\/v2\/media?parent=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}